Uncertainty quantification in mechanistic epidemic models via cross-entropy approximate Bayesian computation

This paper proposes a data-driven approximate Bayesian computation framework for parameter estimation and uncertainty quantification of epidemic models, which incorporates two novelties: (i) the identification of the initial conditions by using plausible dynamic states that are compatible with observational data; (ii) learning of an informative prior distribution for the model parameters via the cross-entropy method. The new methodology's effectiveness is illustrated with the aid of actual data from the COVID-19 epidemic in Rio de Janeiro city in Brazil, employing an ordinary differential equation-based model with a generalized SEIR mechanistic structure that includes time-dependent transmission rate, asymptomatics, and hospitalizations. A minimization problem with two cost terms (number of hospitalizations and deaths) is formulated, and twelve parameters are identified. The calibrated model provides a consistent description of the available data, able to extrapolate forecasts over a few weeks, making the proposed methodology very appealing for real-time epidemic modeling.

Uncertainty and error in SARS-CoV-2 epidemiological parameters inferred from population-level epidemic models

During the SARS-CoV-2 pandemic, epidemic models have been central to policy-making. Public health responses have been shaped by model-based projections and inferences, especially related to the impact of various non-pharmaceutical interventions. Accompanying this has been increased scrutiny over model performance, model assumptions, and the way that uncertainty is incorporated and presented. Here we consider a population-level model, focusing on how distributions representing host infectiousness and the infection-to-death times are modelled, and particularly on the impact of inferred epidemic characteristics if these distributions are mis-specified. We introduce an SIR-type model with the infected population structured by 'infected age', i.e. the number of days since first being infected, a formulation that enables distributions to be incorporated that are consistent with clinical data. We show that inference based on simpler models without infected age, which implicitly mis-specify these distributions, leads to substantial errors in inferred quantities relevant to policy-making, such as the reproduction number and the impact of interventions. We consider uncertainty quantification via a Bayesian approach, implementing this for both synthetic and real data focusing on UK data in the period 15 Feb-14 Jul 2020, and emphasising circumstances where it is misleading to neglect uncertainty. This manuscript was submitted as part of a theme issue on "Modelling COVID-19 and Preparedness for Future Pandemics".

A simple model for viral decay dynamics and the distribution of infected cell life spans in SHIV-infected infant rhesus macaques

The dynamics of HIV viral load following the initiation of antiretroviral therapy is not well-described by simple, single-phase exponential decay. Several mathematical models have been proposed to describe its more complex behavior, the most popular of which is two-phase exponential decay. The underlying assumption in two-phase exponential decay is that there are two classes of infected cells with different lifespans. However, with the exception of CD4+ T cells, there is not a consensus on all of the cell types that can become productively infected, and the fit of the two-phase exponential decay to observed data from SHIV.C.CH505 infected infant rhesus macaques was relatively poor. Therefore, we propose a new model for viral decay, inspired by the Gompertz model where the decay rate itself is a dynamic variable. We modify the Gompertz model to include a linear term that modulates the decay rate. We show that this simple model performs as well as the two-phase exponential decay model on HIV and SIV data sets, and outperforms it for the infant rhesus macaque SHIV.C.CH505 infection data set. We also show that by using a stochastic differential equation formulation, the modified Gompertz model can be interpreted as being driven by a population of infected cells with a continuous distribution of cell lifespans, and estimate this distribution for the SHIV.C.CH505-infected infant rhesus macaques. Thus, we find that the dynamics of viral decay in this model of infant HIV infection and treatment may be explained by a distribution of cell lifespans, rather than two distinct cell types.

Quantifying the impact of shape uncertainty on predicted arrhythmias

BACKGROUND

Personalised computer models are increasingly used to diagnose cardiac arrhythmias and tailor treatment. Patient-specific models of the left atrium are often derived from pre-procedural imaging of anatomy and fibrosis. These images contain noise that can affect simulation predictions. There are few computationally tractable methods for propagating uncertainties from images to clinical predictions.

METHOD

We describe the left atrium anatomy using our Bayesian shape model that captures anatomical uncertainty in medical images and has been validated on 63 independent clinical images. This algorithm describes the left atrium anatomy using Nmodes=15 principal components, capturing 95% of the shape variance and calculated from 70 clinical cardiac magnetic resonance (CMR) images. Latent variables encode shape uncertainty: we evaluate their posterior distribution for each new anatomy. We assume a normally distributed prior. We use the unscented transform to sample from the posterior shape distribution. For each sample, we assign the local material properties of the tissue using the projection of late gadolinium enhancement CMR (LGE-CMR) onto the anatomy to estimate local fibrosis. To test which activation patterns an atrium can sustain, we perform an arrhythmia simulation for each sample. We consider 34 possible outcomes (31 macro-re-entries, functional re-entry, atrial fibrillation, and non-sustained arrhythmia). For each sample, we determine the outcome by comparing pre- and post-ablation activation patterns following a cross-field stimulus.

RESULTS

We create patient-specific atrial electrophysiology models of ten patients. We validate the mean and standard deviation maps from the unscented transform with the same statistics obtained with 12,000 Monte Carlo (ground truth) samples. We found discrepancies <3% and <2% for the mean and standard deviation for fibrosis burden and activation time, respectively. For each patient case, we then compare the predicted outcome from a model built on the clinical data (deterministic approach) with the probability distribution obtained from the simulated samples. We found that the deterministic approach did not predict the most likely outcome in 80% of the cases. Finally, we estimate the influence of each source of uncertainty independently. Fixing the anatomy to the posterior mean and maintaining uncertainty in fibrosis reduced the prediction of self-terminating arrhythmias from ≃14% to ≃7%. Keeping the fibrosis fixed to the sample mean while retaining uncertainty in shape decreased the prediction of substrate-driven arrhythmias from ≃33% to ≃18% and increased the prediction of macro-re-entries from ≃54% to ≃68%.

CONCLUSIONS

We presented a novel method for propagating shape uncertainty in atrial models through to uncertainty in numerical simulations. The algorithm takes advantage of the unscented transform to compute the output distribution of the outcomes. We validated the unscented transform as a viable sampling strategy to deal with anatomy uncertainty. We then showed that the prediction computed with a deterministic model does not always coincide with the most likely outcome. Finally, we found that shape uncertainty affects the predictions of macro-re-entries, while fibrosis uncertainty affects the predictions of functional re-entries.

UncertaintyFuseNet: Robust uncertainty-aware hierarchical feature fusion model with Ensemble Monte Carlo Dropout for COVID-19 detection

The COVID-19 (Coronavirus disease 2019) pandemic has become a major global threat to human health and well-being. Thus, the development of computer-aided detection (CAD) systems that are capable of accurately distinguishing COVID-19 from other diseases using chest computed tomography (CT) and X-ray data is of immediate priority. Such automatic systems are usually based on traditional machine learning or deep learning methods. Differently from most of the existing studies, which used either CT scan or X-ray images in COVID-19-case classification, we present a new, simple but efficient deep learning feature fusion model, called U n c e r t a i n t y F u s e N e t , which is able to classify accurately large datasets of both of these types of images. We argue that the uncertainty of the model's predictions should be taken into account in the learning process, even though most of the existing studies have overlooked it. We quantify the prediction uncertainty in our feature fusion model using effective Ensemble Monte Carlo Dropout (EMCD) technique. A comprehensive simulation study has been conducted to compare the results of our new model to the existing approaches, evaluating the performance of competing models in terms of Precision, Recall, F-Measure, Accuracy and ROC curves. The obtained results prove the efficiency of our model which provided the prediction accuracy of 99.08% and 96.35% for the considered CT scan and X-ray datasets, respectively. Moreover, our U n c e r t a i n t y F u s e N e t model was generally robust to noise and performed well with previously unseen data. The source code of our implementation is freely available at: https://github.com/moloud1987/UncertaintyFuseNet-for-COVID-19-Classification.

Uncertainty quantification in DenseNet model using myocardial infarction ECG signals

BACKGROUND AND OBJECTIVE

Myocardial infarction (MI) is a life-threatening condition diagnosed acutely on the electrocardiogram (ECG). Several errors, such as noise, can impair the prediction of automated ECG diagnosis. Therefore, quantification and communication of model uncertainty are essential for reliable MI diagnosis.

METHODS

A Dirichlet DenseNet model that could analyze out-of-distribution data and detect misclassification of MI and normal ECG signals was developed. The DenseNet model was first trained with the pre-processed MI ECG signals (from the best lead V6) acquired from the Physikalisch-Technische Bundesanstalt (PTB) database, using the reverse Kullback-Leibler (KL) divergence loss. The model was then tested with newly synthesized ECG signals with added em and ma noise samples. Predictive entropy was used as an uncertainty measure to determine the misclassification of normal and MI signals. Model performance was evaluated using four uncertainty metrics: uncertainty sensitivity (UNSE), uncertainty specificity (UNSP), uncertainty accuracy (UNAC), and uncertainty precision (UNPR); the classification threshold was set at 0.3.

RESULTS

The UNSE of the DenseNet model was low but increased over the studied decremental noise range (-6 to 24 dB), indicating that the model grew more confident in classifying the signals as they got less noisy. The model became more certain in its predictions from SNR values of 12 dB and 18 dB onwards, yielding UNAC values of 80% and 82.4% for em and ma noise signals, respectively. UNSP and UNPR values were close to 100% for em and ma noise signals, indicating that the model was self-aware of what it knew and didn't.

CONCLUSION

Through this work, it has been established that the model is reliable as it was able to convey when it was not confident in the diagnostic information it was presenting. Thus, the model is trustworthy and can be used in healthcare applications, such as the emergency diagnosis of MI on ECGs.

Industrial ion-exchange chromatography development using discontinuous Galerkin methods coupled with forward sensitivity analysis

In this work, a discontinuous Galerkin method coupled with forward sensitivity analysis (DG-FSA) is presented. The DG-FSA method is used to reduce computational cost required for model-based ion-exchange chromatography development using industrial load samples. As an example, the design of an anion-exchange chromatography step is considered. This step is used to purify an experimental peptide product called Protein G from Novo Nordisk A/S (Bagsværd, Denmark). The results demonstrate, that a fourth order DG-FSA method can reduce computational cost of inverse problems by a factor ×16 compared to a second (low) order DG-FSA method. Furthermore, the fourth-order DG-FSA method enable the computation of probability distributions of optimized processing conditions given uncertainty in model parameters or inputs. This analysis is not possible within a reasonable timeframe when applying the second (low) order DG-FSA method. The design procedure facilitates the optimization of the Protein G purification step. In an experimental validation run, the productivity is increased by 70% while sacrificing 4% yield at a similar purity constraint compared to an experiment with baseline performance.

Calibration of Cohorts of Virtual Patient Heart Models Using Bayesian History Matching

Previous patient-specific model calibration techniques have treated each patient independently, making the methods expensive for large-scale clinical adoption. In this work, we show how we can reuse simulations to accelerate the patient-specific model calibration pipeline. To represent anatomy, we used a Statistical Shape Model and to represent function, we ran electrophysiological simulations. We study the use of 14 biomarkers to calibrate the model, training one Gaussian Process Emulator (GPE) per biomarker. To fit the models, we followed a Bayesian History Matching (BHM) strategy, wherein each iteration a region of the parameter space is ruled out if the emulation with that set of parameter values produces is "implausible". We found that without running any extra simulations we can find 87.41% of the non-implausible parameter combinations. Moreover, we showed how reducing the uncertainty of the measurements from 10 to 5% can reduce the final parameter space by 6 orders of magnitude. This innovation allows for a model fitting technique, therefore reducing the computational load of future biomedical studies.

A Critical Comparison of Comparators Used to Demonstrate Credibility of Physics-Based Numerical Spine Models

The ability of new medical devices and technology to demonstrate safety and effectiveness, and consequently acquire regulatory approval, has been dependent on benchtop, in vitro, and in vivo evidence and experimentation. Regulatory agencies have recently begun accepting computational models and simulations as credible evidence for virtual clinical trials and medical device development. However, it is crucial that any computational model undergo rigorous verification and validation activities to attain credibility for its context of use before it can be accepted for regulatory submission. Several recently published numerical models of the human spine were considered for their implementation of various comparators as a means of model validation. The comparators used in each published model were examined and classified as either an engineering or natural comparator. Further, a method of scoring the comparators was developed based on guidelines from ASME V&V40 and the draft guidance from the US FDA, and used to evaluate the pertinence of each comparator in model validation. Thus, this review article aimed to score the various comparators used to validate numerical models of the spine in order to examine the comparator's ability to lend credibility towards computational models of the spine for specific contexts of use.

UncertainSCI: Uncertainty quantification for computational models in biomedicine and bioengineering

BACKGROUND

Computational biomedical simulations frequently contain parameters that model physical features, material coefficients, and physiological effects, whose values are typically assumed known a priori. Understanding the effect of variability in those assumed values is currently a topic of great interest. A general-purpose software tool that quantifies how variation in these parameters affects model outputs is not broadly available in biomedicine. For this reason, we developed the 'UncertainSCI' uncertainty quantification software suite to facilitate analysis of uncertainty due to parametric variability.

METHODS

We developed and distributed a new open-source Python-based software tool, UncertainSCI, which employs advanced parameter sampling techniques to build polynomial chaos (PC) emulators that can be used to predict model outputs for general parameter values. Uncertainty of model outputs is studied by modeling parameters as random variables, and model output statistics and sensitivities are then easily computed from the emulator. Our approaches utilize modern, near-optimal techniques for sampling and PC construction based on weighted Fekete points constructed by subsampling from a suitably randomized candidate set.

RESULTS

Concentrating on two test cases-modeling bioelectric potentials in the heart and electric stimulation in the brain-we illustrate the use of UncertainSCI to estimate variability, statistics, and sensitivities associated with multiple parameters in these models.

CONCLUSION

UncertainSCI is a powerful yet lightweight tool enabling sophisticated probing of parametric variability and uncertainty in biomedical simulations. Its non-intrusive pipeline allows users to leverage existing software libraries and suites to accurately ascertain parametric uncertainty in a variety of applications.

Uncertainty quantification of PM2.5 concentrations using a hybrid model based on characteristic decomposition and fuzzy granulation

The prediction of air pollution plays an important role in reducing the emission of air pollutants and guiding people to carry out early warning and control, so it attracts many scholars to conduct modeling and research on it. However, most of the current researches fail to quantify the uncertainty in prediction and only use traditional fuzzy information granulation to process data, resulting in the loss of much detail information. Therefore, this paper proposes a hybrid model based on decomposition and granular fuzzy information to solve these problems. The trend item and the Granulation fluctuation item are respectively predicted and the results are combined to obtain the change trend and fluctuation range of the sequence. This paper selects PM_2.5 concentrations of 3 cities. The experimental results show that the evaluation index of the prediction model is significantly lower than other benchmark models, and a variety of statistical methods are used to further verify the effectiveness of the prediction model.

Development of a DREAM-based inverse model for multi-point source identification in river pollution incidents: Model testing and uncertainty analysis

Source identification plays a vital role in implementing control measures for sudden river pollution incidents. In contrast to single-point source identification problems, there have been no investigations into inverse identification of multi-point emissions. In this study, an inverse model is developed based on the observed time series of pollutant concentrations and the DiffeRential Evolution Adaptive Metropolis (DREAM) method to identify multi-point sources with uncertainty quantification. We aim to simultaneously determine source mass, release location and time of multi-point sources. The newly developed DREAM-based model has been tested and verified through both numerical and field data case studies in terms of accuracy, reliability, and computational time. Adapted cases with single-point, two-point and three-point sources in the Songhua River are conducted to test the applicability of the modeling approach, respectively. The developed model can correctly quantify source parameters with a relative error that does not exceed ±0.63%, although it shows that an increase of emission sources may slightly increase the identification error. Among the three source parameters, the identification error of the release time tends to rise more obviously in response to the increase in the number of pollution sources. It is also found that the identification accuracy is primarily sensitive to the river velocity, followed by the dispersion coefficient and the river cross-sectional area. Furthermore, good monitoring strategies, including reducing observation errors, shortening monitoring interval time and selecting the proper monitoring distance between the monitoring and the source sites, help to achieve a better application of the developed model in river pollution incidents.

Predicting in-stream water quality constituents at the watershed scale using machine learning

Predicting in-stream water quality is necessary to support the decision-making process of protecting healthy waterbodies and restoring impaired ones. Data-driven modeling is an efficient technique that can be used to support such efforts. Our objective was to determine if in-stream concentrations of contaminants, nutrients-total phosphorus (TP) and total nitrogen (TN) -total suspended solids (TSS), dissolved oxygen (DO), and fecal coliform bacteria (FC) can be predicted satisfactorily using machine learning (ML) algorithms based on publicly available datasets. To achieve this objective, we evaluated four modeling scenarios, differing in terms of the required inputs (i.e., publicly available datasets (e.g., land-use/land cover)), antecedent conditions, and additional in-stream water quality observations (e.g., pH and turbidity). We implemented five ML algorithms-Support Vector Machines, Random Forest (RF), eXtreme Gradient Boost (XGB), ensemble RF-XGB, and Artificial Neural Network (ANN) -and demonstrated our modeling framework in an inland stream-Bullfrog Creek, located near Tampa, Florida. The results showed that, while including additional water quality drivers improved overall model performance for all target constituents, TP, TN, DO, and TSS could still be predicted satisfactorily using only publicly available datasets (Nash-Sutcliffe efficiency [NSE] > 0.75 and percent bias [PBIAS] < 10%), whereas FC could not (NSE < 0.49 and PBIAS >25%). Additionally, antecedent conditions slightly improved predictions and reduced the predictive uncertainty, particularly when paired with other water quality observations (6.9% increase in NSE for FC, and 2.7% for TP, TN, DO, and TSS). Also, comparable model performances of all water quality constituents in wet and dry seasons suggest minimal season-dependence of the predictions (<4% difference in NSE and < 10% difference in PBIAS). Our developed modeling framework is generic and can serve as a complementary tool for monitoring and predicting in-stream water quality constituents.

Optimizing combination therapy in a murine model of HER2+ breast cancer

Human epidermal growth factor receptor 2 positive (HER2+) breast cancer is frequently treated with drugs that target the HER2 receptor, such as trastuzumab, in combination with chemotherapy, such as doxorubicin. However, an open problem in treatment design is to determine the therapeutic regimen that optimally combines these two treatments to yield optimal tumor control. Working with data quantifying temporal changes in tumor volume due to different trastuzumab and doxorubicin treatment protocols in a murine model of human HER2+ breast cancer, we propose a complete framework for model development, calibration, selection, and treatment optimization to find the optimal treatment protocol. Through different assumptions for the drug-tumor interactions, we propose ten different models to characterize the dynamic relationship between tumor volume and drug availability, as well as the drug-drug interaction. Using a Bayesian framework, each of these models are calibrated to the dataset and the model with the highest Bayesian information criterion weight is selected to represent the biological system. The selected model captures the inhibition of trastuzumab due to pre-treatment with doxorubicin, as well as the increase in doxorubicin efficacy due to pre-treatment with trastuzumab. We then apply optimal control theory (OCT) to this model to identify two optimal treatment protocols. In the first optimized protocol, we fix the maximum dosage for doxorubicin and trastuzumab to be the same as the maximum dose delivered experimentally, while trying to minimize tumor burden. Within this constraint, optimal control theory indicates the optimal regimen is to first deliver two doses of trastuzumab on days 35 and 36, followed by two doses of doxorubicin on days 37 and 38. This protocol predicts an additional 45% reduction in tumor burden compared to that achieved with the experimentally delivered regimen. In the second optimized protocol we fix the tumor control to be the same as that obtained experimentally, and attempt to reduce the doxorubicin dose. Within this constraint, the optimal regimen is the same as the first optimized protocol but uses only 43% of the doxorubicin dose used experimentally. This protocol predicts tumor control equivalent to that achieved experimentally. These results strongly suggest the utility of mathematical modeling and optimal control theory for identifying therapeutic regimens maximizing efficacy and minimizing toxicity.

Region-based evidential deep learning to quantify uncertainty and improve robustness of brain tumor segmentation

Despite recent advances in the accuracy of brain tumor segmentation, the results still suffer from low reliability and robustness. Uncertainty estimation is an efficient solution to this problem, as it provides a measure of confidence in the segmentation results. The current uncertainty estimation methods based on quantile regression, Bayesian neural network, ensemble, and Monte Carlo dropout are limited by their high computational cost and inconsistency. In order to overcome these challenges, Evidential Deep Learning (EDL) was developed in recent work but primarily for natural image classification and showed inferior segmentation results. In this paper, we proposed a region-based EDL segmentation framework that can generate reliable uncertainty maps and accurate segmentation results, which is robust to noise and image corruption. We used the Theory of Evidence to interpret the output of a neural network as evidence values gathered from input features. Following Subjective Logic, evidence was parameterized as a Dirichlet distribution, and predicted probabilities were treated as subjective opinions. To evaluate the performance of our model on segmentation and uncertainty estimation, we conducted quantitative and qualitative experiments on the BraTS 2020 dataset. The results demonstrated the top performance of the proposed method in quantifying segmentation uncertainty and robustly segmenting tumors. Furthermore, our proposed new framework maintained the advantages of low computational cost and easy implementation and showed the potential for clinical application.

A Bayesian deep learning method for freeway incident detection with uncertainty quantification

Incident detection is fundamental for freeway management to reduce non-recurrent congestions and secondary incidents. Recently, machine learning technologies have made considerable progress in the incident detection field, but many still face challenges in uncertainty quantification due to the aleatoric uncertainty of traffic data and the epistemic uncertainty of model deviations. In this study, a Bayesian deep learning method was proposed for freeway incident detection with uncertainty quantification. A convolutional neural network variant was designed on a Bayesian framework, and mechanisms of Bayes by backpropagation and local reparameterization technics were used to update the weight of the proposed model. The predictive uncertainty of the proposed method was modeled jointly by integrating the aleatoric and epistemic uncertainty. The proposed model was tested on the PORTAL dataset and compared with four benchmark models: standard normal deviate, wavelet neural network, long-short term memory neural network, and convolutional neural network. The results show that the proposed model outperforms the baseline methods in terms of accuracy, detection rate and false alarm rate. Perturbation experiments were used to test the robustness of the model against noise. The results indicated that the aleatoric uncertainty of the model remained almost constant under different noise levels. The proposed method may benefit future studies on uncertainty quantification while using machine learning method in incident management and other fields in intelligent transportation systems.

Information entropy tradeoffs for efficient uncertainty reduction in estimates of air pollution mortality

Implementing effective policy to protect human health from the adverse effects of air pollution, such as premature mortality, requires reducing the uncertainty in health outcomes models. Here we present a novel method to reduce mortality uncertainty by increasing the amount of input data of air pollution and health outcomes, and then quantifying tradeoffs associated with the different data gained. We first present a study of long-term mortality from fine particulate matter (PM_2.5) based on simulated data, followed by a real-world application of short-term PM_2.5-related mortality in an urban area. We employ information yield curves to identify which variables more effectively reduce mortality uncertainty when increasing information. Our methodology can be used to explore how specific pollution scenarios will impact mortality and thus improve decision-making. The proposed framework is general and can be applied to any real case-scenario where knowledge in pollution, demographics, or health outcomes can be augmented through data acquisition or model improvements to generate more robust risk assessments.

Dynamic Linear Transformer for 3D Biomedical Image Segmentation

Transformer-based neural networks have surpassed promising performance on many biomedical image segmentation tasks due to a better global information modeling from the self-attention mechanism. However, most methods are still designed for 2D medical images while ignoring the essential 3D volume information. The main challenge for 3D Transformer-based segmentation methods is the quadratic complexity introduced by the self-attention mechanism [17]. In this paper, we are addressing these two research gaps, lack of 3D methods and computational complexity in Transformers, by proposing a novel Transformer architecture that has an encoder-decoder style architecture with linear complexity. Furthermore, we newly introduce a dynamic token concept to further reduce the token numbers for self-attention calculation. Taking advantage of the global information modeling, we provide uncertainty maps from different hierarchy stages. We evaluate this method on multiple challenging CT pancreas segmentation datasets. Our results show that our novel 3D Transformer-based segmentor could provide promising highly feasible segmentation performance and accurate uncertainty quantification using single annotation. Code is available https://github.com/freshman97/LinTransUNet.

Conformal Prediction in Clinical Medical Sciences

The use of machine learning (ML) and artificial intelligence (AI) applications in medicine has attracted a great deal of attention in the medical literature, but little is known about how to use Conformal Predictions (CP) to assess the accuracy of individual predictions in clinical applications. We performed a comprehensive search in SCOPUS® to find papers reporting the use of CP in clinical applications. We identified 14 papers reporting the use of CP for clinical applications, and we briefly describe the methods and results reported in these papers. The literature reviewed shows that CP methods can be used in clinical applications to provide important insight into the accuracy of individual predictions. Unfortunately, the review also shows that most of the studies have been performed in isolation, without input from practicing clinicians, not providing comparisons among different approaches and not considering important socio-technical considerations leading to clinical adoption.

Uncertainty quantification of atmospheric transport and dispersion modelling using ensembles for CTBT verification applications

Airborne concentrations of specific radioactive xenon isotopes (referred to as "radioxenon") are monitored globally as part of the verification regime of the Comprehensive Nuclear-Test-Ban Treaty, as these could be the signatures of a nuclear explosion. However, civilian nuclear facilities emit a regulated amount of radioxenon that can interfere with the very sensitive monitoring network. One approach to deal with this civilian background of radioxenon for Treaty verification purposes, is to explicitly simulate the expected radioxenon concentration from civilian sources at monitoring stations using atmospheric transport modelling. However, atmospheric transport modelling is prone to uncertainty, and the absence of an uncertainty quantification can limit its use for detection screening. In this paper, several ensembles are assessed that could provide an atmospheric transport modelling uncertainty quantification. These ensembles are validated with radioxenon observations, and recommendations are given for atmospheric transport modelling uncertainty quantification. Finally, the added value of an ensemble for detection screening is illustrated.

Development and evaluation of uncertainty quantifying machine learning models to predict piperacillin plasma concentrations in critically ill patients

BACKGROUND

Beta-lactam antimicrobial concentrations are frequently suboptimal in critically ill patients. Population pharmacokinetic (PopPK) modeling is the golden standard to predict drug concentrations. However, currently available PopPK models often lack predictive accuracy, making them less suited to guide dosing regimen adaptations. Furthermore, many currently developed models for clinical applications often lack uncertainty quantification. We, therefore, aimed to develop machine learning (ML) models for the prediction of piperacillin plasma concentrations while also providing uncertainty quantification with the aim of clinical practice.

METHODS

Blood samples for piperacillin analysis were prospectively collected from critically ill patients receiving continuous infusion of piperacillin/tazobactam. Interpretable ML models for the prediction of piperacillin concentrations were designed using CatBoost and Gaussian processes. Distribution-based Uncertainty Quantification was added to the CatBoost model using a proposed Quantile Ensemble method, useable for any model optimizing a quantile function. These models are subsequently evaluated using the distribution coverage error, a proposed interpretable uncertainty quantification calibration metric. Development and internal evaluation of the ML models were performed on the Ghent University Hospital database (752 piperacillin concentrations from 282 patients). Ensuing, ML models were compared with a published PopPK model on a database from the University Medical Centre of Groningen where a different dosing regimen is used (46 piperacillin concentrations from 15 patients.).

RESULTS

The best performing model was the Catboost model with an RMSE and [Formula: see text] of 31.94-0.64 and 33.53-0.60 for internal evaluation with and without previous concentration. Furthermore, the results prove the added value of the proposed Quantile Ensemble model in providing clinically useful individualized uncertainty predictions and show the limits of homoscedastic methods like Gaussian Processes in clinical applications.

CONCLUSIONS

Our results show that ML models can consistently estimate piperacillin concentrations with acceptable and high predictive accuracy when identical dosing regimens as in the training data are used while providing highly relevant uncertainty predictions. However, generalization capabilities to other dosing schemes are limited. Notwithstanding, incorporating ML models in therapeutic drug monitoring programs seems definitely promising and the current work provides a basis for validating the model in clinical practice.

Estimating uncertainty in density surface models

Providing uncertainty estimates for predictions derived from species distribution models is essential for management but there is little guidance on potential sources of uncertainty in predictions and how best to combine these. Here we show where uncertainty can arise in density surface models (a multi-stage spatial modelling approach for distance sampling data), focussing on cetacean density modelling. We propose an extensible, modular, hybrid analytical-simulation approach to encapsulate these sources. We provide example analyses of fin whales Balaenoptera physalus in the California Current Ecosystem.

Hyper-differential sensitivity analysis for inverse problems governed by ODEs with application to COVID-19 modeling

We consider inverse problems governed by systems of ordinary differential equations (ODEs) that contain uncertain parameters in addition to the parameters being estimated. In such problems, which are common in applications, it is important to understand the sensitivity of the solution of the inverse problem to the uncertain model parameters. It is also of interest to understand the sensitivity of the inverse problem solution to different types of measurements or parameters describing the experimental setup. Hyper-differential sensitivity analysis (HDSA) is a sensitivity analysis approach that provides tools for such tasks. We extend existing HDSA methods by developing methods for quantifying the uncertainty in the estimated parameters. Specifically, we propose a linear approximation to the solution of the inverse problem that allows efficiently approximating the statistical properties of the estimated parameters. We also explore the use of this linear model for approximate global sensitivity analysis. As a driving application, we consider an inverse problem governed by a COVID–19 model. We present comprehensive computational studies that examine the sensitivity of this inverse problem to several uncertain model parameters and different types of measurement data. Our results also demonstrate the effectiveness of the linear approximation model for uncertainty quantification in inverse problems and for parameter screening.

Probabilistic solvers enable a straight-forward exploration of numerical uncertainty in neuroscience models

Understanding neural computation on the mechanistic level requires models of neurons and neuronal networks. To analyze such models one typically has to solve coupled ordinary differential equations (ODEs), which describe the dynamics of the underlying neural system. These ODEs are solved numerically with deterministic ODE solvers that yield single solutions with either no, or only a global scalar error indicator on precision. It can therefore be challenging to estimate the effect of numerical uncertainty on quantities of interest, such as spike-times and the number of spikes. To overcome this problem, we propose to use recently developed sampling-based probabilistic solvers, which are able to quantify such numerical uncertainties. They neither require detailed insights into the kinetics of the models, nor are they difficult to implement. We show that numerical uncertainty can affect the outcome of typical neuroscience simulations, e.g. jittering spikes by milliseconds or even adding or removing individual spikes from simulations altogether, and demonstrate that probabilistic solvers reveal these numerical uncertainties with only moderate computational overhead.

Data-driven Uncertainty Quantification in Computational Human Head Models

Computational models of the human head are promising tools for estimating the impact-induced response of the brain, and thus play an important role in the prediction of traumatic brain injury. The basic constituents of these models (i.e., model geometry, material properties, and boundary conditions) are often associated with significant uncertainty and variability. As a result, uncertainty quantification (UQ), which involves quantification of the effect of this uncertainty and variability on the simulated response, becomes critical to ensure reliability of model predictions. Modern biofidelic head model simulations are associated with very high computational cost and high-dimensional inputs and outputs, which limits the applicability of traditional UQ methods on these systems. In this study, a two-stage, data-driven manifold learning-based framework is proposed for UQ of computational head models. This framework is demonstrated on a 2D subject-specific head model, where the goal is to quantify uncertainty in the simulated strain fields (i.e., output), given variability in the material properties of different brain substructures (i.e., input). In the first stage, a data-driven method based on multi-dimensional Gaussian kernel-density estimation and diffusion maps is used to generate realizations of the input random vector directly from the available data. Computational simulations of a small number of realizations provide input-output pairs for training data-driven surrogate models in the second stage. The surrogate models employ nonlinear dimensionality reduction using Grassmannian diffusion maps, Gaussian process regression to create a low-cost mapping between the input random vector and the reduced solution space, and geometric harmonics models for mapping between the reduced space and the Grassmann manifold. It is demonstrated that the surrogate models provide highly accurate approximations of the computational model while significantly reducing the computational cost. Monte Carlo simulations of the surrogate models are used for uncertainty propagation. UQ of the strain fields highlights significant spatial variation in model uncertainty, and reveals key differences in uncertainty among commonly used strain-based brain injury predictor variables.