Bayesian model assessment and comparison using cross-validation predictive densities

The successor representation subserves hierarchical abstraction for goal-directed behavior

Humans have the ability to craft abstract, temporally extended and hierarchically organized plans. For instance, when considering how to make spaghetti for dinner, we typically concern ourselves with useful "subgoals" in the task, such as cutting onions, boiling pasta, and cooking a sauce, rather than particulars such as how many cuts to make to the onion, or exactly which muscles to contract. A core question is how such decomposition of a more abstract task into logical subtasks happens in the first place. Previous research has shown that humans are sensitive to a form of higher-order statistical learning named "community structure". Community structure is a common feature of abstract tasks characterized by a logical ordering of subtasks. This structure can be captured by a model where humans learn predictions of upcoming events multiple steps into the future, discounting predictions of events further away in time. One such model is the "successor representation", which has been argued to be useful for hierarchical abstraction. As of yet, no study has convincingly shown that this hierarchical abstraction can be put to use for goal-directed behavior. Here, we investigate whether participants utilize learned community structure to craft hierarchically informed action plans for goal-directed behavior. Participants were asked to search for paintings in a virtual museum, where the paintings were grouped together in "wings" representing community structure in the museum. We find that participants' choices accord with the hierarchical structure of the museum and that their response times are best predicted by a successor representation. The degree to which the response times reflect the community structure of the museum correlates with several measures of performance, including the ability to craft temporally abstract action plans. These results suggest that successor representation learning subserves hierarchical abstractions relevant for goal-directed behavior.

Deep learning-based overall survival prediction model in patients with rare cancer: a case study for primary central nervous system lymphoma

PURPOSE

Primary central nervous system lymphoma (PCNSL) is a rare, aggressive form of extranodal non-Hodgkin lymphoma. To predict the overall survival (OS) in advance is of utmost importance as it has the potential to aid clinical decision-making. Though radiomics-based machine learning (ML) has demonstrated the promising performance in PCNSL, it demands large amounts of manual feature extraction efforts from magnetic resonance images beforehand. deep learning (DL) overcomes this limitation.

METHODS

In this paper, we tailored the 3D ResNet to predict the OS of patients with PCNSL. To overcome the limitation of data sparsity, we introduced data augmentation and transfer learning, and we evaluated the results using r stratified k-fold cross-validation. To explain the results of our model, gradient-weighted class activation mapping was applied.

RESULTS

We obtained the best performance (the standard error) on post-contrast T1-weighted (T1Gd)-area under curve [Formula: see text], accuracy [Formula: see text], precision [Formula: see text], recall [Formula: see text] and F1-score [Formula: see text], while compared with ML-based models on clinical data and radiomics data, respectively, further confirming the stability of our model. Also, we observed that PCNSL is a whole-brain disease and in the cases where the OS is less than 1 year, it is more difficult to distinguish the tumor boundary from the normal part of the brain, which is consistent with the clinical outcome.

CONCLUSIONS

All these findings indicate that T1Gd can improve prognosis predictions of patients with PCNSL. To the best of our knowledge, this is the first time to use DL to explain model patterns in OS classification of patients with PCNSL. Future work would involve collecting more data of patients with PCNSL, or additional retrospective studies on different patient populations with rare diseases, to further promote the clinical role of our model.

Hybrid BO-XGBoost and BO-RF Models for the Strength Prediction of Self-Compacting Mortars with Parametric Analysis

Self-compacting mortar (SCM) has superior workability and long-term durable performance compared to traditional mortar. The strength of SCM, including both its compressive and flexural strengths, is a crucial property that is determined by appropriate curing conditions and mix design parameters. In the context of materials science, predicting the strength of SCM is challenging because of multiple influencing factors. This study employed machine learning techniques to establish SCM strength prediction models. Based on ten different input parameters, the strength of SCM specimens were predicted using two different types of hybrid machine learning (HML) models, namely Extreme Gradient Boosting (XGBoost) and the Random Forest (RF) algorithm. HML models were trained and tested by experimental data from 320 test specimens. In addition, the Bayesian optimization method was utilized to fine tune the hyperparameters of the employed algorithms, and cross-validation was employed to partition the database into multiple folds for a more thorough exploration of the hyperparameter space while providing a more accurate assessment of the model's predictive power. The results show that both HML models can successfully predict the SCM strength values with high accuracy, and the Bo-XGB model demonstrated higher accuracy (R² = 0.96 for training and R² = 0.91 for testing phases) for predicting flexural strength with low error. In terms of compressive strength prediction, the employed BO-RF model performed very well, with R² = 0.96 for train and R² = 0.88 testing stages with minor errors. Moreover, the SHAP algorithm, permutation importance and leave-one-out importance score were used for sensitivity analysis to explain the prediction process and interpret the governing input variable parameters of the proposed HML models. Finally, the outcomes of this study might be applied to guide the future mix design of SCM specimens.

Mathematical theory of Bayesian statistics for unknown information source

In statistical inference, uncertainty is unknown and all models are wrong. That is to say, a person who makes a statistical model and a prior distribution is simultaneously aware that both are fictional candidates. To study such cases, statistical measures have been constructed, such as cross validation, information criteria and marginal likelihood; however, their mathematical properties have not yet been completely clarified when statistical models are under- or over-parametrized. We introduce a place of mathematical theory of Bayesian statistics for unknown uncertainty, which clarifies general properties of cross validation, information criteria and marginal likelihood, even if an unknown data-generating process is unrealizable by a model or even if the posterior distribution cannot be approximated by any normal distribution. Hence it gives a helpful standpoint for a person who cannot believe in any specific model and prior. This paper consists of three parts. The first is a new result, whereas the second and third are well-known previous results with new experiments. We show there exists a more precise estimator of the generalization loss than leave-one-out cross validation, there exists a more accurate approximation of marginal likelihood than Bayesian information criterion, and the optimal hyperparameters for generalization loss and marginal likelihood are different. This article is part of the theme issue 'Bayesian inference: challenges, perspectives, and prospects'.

Posterior Averaging Information Criterion

We propose a new model selection method, named the posterior averaging information criterion, for Bayesian model assessment to minimize the risk of predicting independent future observations. The theoretical foundation is built on the Kullback-Leibler divergence to quantify the similarity between the proposed candidate model and the underlying true model. From a Bayesian perspective, our method evaluates the candidate models over the entire posterior distribution in terms of predicting a future independent observation. Without assuming that the true distribution is contained in the candidate models, the new criterion is developed by correcting the asymptotic bias of the posterior mean of the in-sample log-likelihood against out-of-sample log-likelihood, and can be generally applied even for Bayesian models with degenerate non-informative priors. Simulations in both normal and binomial settings demonstrate superior small sample performance.

Hedonism as a motive for information search: biased information-seeking leads to biased beliefs

Confirmation bias in information-search contributes to the formation of polarized echo-chambers of beliefs. However, the role of valence on information source selection remains poorly understood. In Experiment 1, participants won financial rewards depending on the outcomes of a set of lotteries. They were not shown these outcomes, but instead could choose to view a prediction of each lottery outcome made by one of two sources. Before choosing their favoured source, participants were first shown a series of example predictions made by each. The sources systematically varied in the accuracy and positivity (i.e., how often they predicted a win) of their predictions. Hierarchical Bayesian modeling indicated that both source accuracy and positivity impacted participants' choices. Importantly, those that viewed more positively-biased information believed that they had won more often and had higher confidence in those beliefs. In Experiment 2, we directly assessed the effect of positivity on the perceived credibility of a source. In each trial, participants watched a single source making a series of predictions of lottery outcomes and rated the strength of their beliefs in each source. Interestingly, positively-biased sources were not seen as more credible. Together, these findings suggest that positively-biased information is sought partly due to the desirable emotional state it induces rather than having enhanced perceived credibility. Information sought on this basis nevertheless produced consequential biased beliefs about the world-state, highlighting a potentially key role for hedonic preferences in information selection and subsequent belief formation.

Information theoretic evidence for layer- and frequency-specific changes in cortical information processing under anesthesia

Nature relies on highly distributed computation for the processing of information in nervous systems across the entire animal kingdom. Such distributed computation can be more easily understood if decomposed into the three elementary components of information processing, i.e. storage, transfer and modification, and rigorous information theoretic measures for these components exist. However, the distributed computation is often also linked to neural dynamics exhibiting distinct rhythms. Thus, it would be beneficial to associate the above components of information processing with distinct rhythmic processes where possible. Here we focus on the storage of information in neural dynamics and introduce a novel spectrally-resolved measure of active information storage (AIS). Drawing on intracortical recordings of neural activity in ferrets under anesthesia before and after loss of consciousness (LOC) we show that anesthesia- related modulation of AIS is highly specific to different frequency bands and that these frequency-specific effects differ across cortical layers and brain regions. We found that in the high/low gamma band the effects of anesthesia result in AIS modulation only in the supergranular layers, while in the alpha/beta band the strongest decrease in AIS can be seen at infragranular layers. Finally, we show that the increase of spectral power at multiple frequencies, in particular at alpha and delta bands in frontal areas, that is often observed during LOC ('anteriorization') also impacts local information processing-but in a frequency specific way: Increases in isoflurane concentration induced a decrease in AIS in the alpha frequencies, while they increased AIS in the delta frequency range < 2Hz. Thus, the analysis of spectrally-resolved AIS provides valuable additional insights into changes in cortical information processing under anaesthesia.

Towards a Stochastic Model to Simulate Grapevine Architecture: A Case Study on Digitized Riesling Vines Considering Effects of Elevated CO2

Modeling plant growth, in particular with functional-structural plant models, can provide tools to study impacts of changing environments in silico. Simulation studies can be used as pilot studies for reducing the on-field experimental effort when predictive capabilities are given. Robust model calibration leads to less fragile predictions, while introducing uncertainties in predictions allows accounting for natural variability, resulting in stochastic plant growth models. In this study, stochastic model components that can be implemented into the functional-structural plant model Virtual Riesling are developed relying on Bayesian model calibration with the goal to enhance the model towards a fully stochastic model. In this first step, model development targeting phenology, in particular budburst variability, phytomer development rate and internode growth are presented in detail. Multi-objective optimization is applied to estimate a single set of cardinal temperatures, which is used in phenology and growth modeling based on a development days approach. Measurements from two seasons of grapevines grown in a vineyard with free-air carbon dioxide enrichment (FACE) are used; thus, model building and selection are coupled with an investigation as to whether including effects of elevated CO2 conditions to be expected in 2050 would improve the models. The results show how natural variability complicates the detection of possible treatment effects, but demonstrate that Bayesian calibration in combination with mixed models can realistically recover natural shoot growth variability in predictions. We expect these and further stochastic model extensions to result in more realistic virtual plant simulations to study effects, which are used to conduct in silico studies of canopy microclimate and its effects on grape health and quality.

Greed Is Good: Rapid Hyperparameter Optimization and Model Selection Using Greedy k-Fold Cross Validation

Selecting a final machine learning (ML) model typically occurs after a process of hyperparameter optimization in which many candidate models with varying structural properties and algorithmic settings are evaluated and compared. Evaluating each candidate model commonly relies on k-fold cross validation, wherein the data are randomly subdivided into k folds, with each fold being iteratively used as a validation set for a model that has been trained using the remaining folds. While many research studies have sought to accelerate ML model selection by applying metaheuristic and other search methods to the hyperparameter space, no consideration has been given to the k-fold cross validation process itself as a means of rapidly identifying the best-performing model. The current study rectifies this oversight by introducing a greedy k-fold cross validation method and demonstrating that greedy k-fold cross validation can vastly reduce the average time required to identify the best-performing model when given a fixed computational budget and a set of candidate models. This improved search time is shown to hold across a variety of ML algorithms and real-world datasets. For scenarios without a computational budget, this paper also introduces an early stopping algorithm based on the greedy cross validation method. The greedy early stopping method is shown to outperform a competing, state-of-the-art early stopping method both in terms of search time and the quality of the ML models selected by the algorithm. Since hyperparameter optimization is among the most time-consuming, computationally intensive, and monetarily expensive tasks in the broader process of developing ML-based solutions, the ability to rapidly identify optimal machine learning models using greedy cross validation has obvious and substantial benefits to organizations and researchers alike.

On the influence of prior information evaluated by fully Bayesian criteria in a personalized whole-brain model of epilepsy spread

Individualized anatomical information has been used as prior knowledge in Bayesian inference paradigms of whole-brain network models. However, the actual sensitivity to such personalized information in priors is still unknown. In this study, we introduce the use of fully Bayesian information criteria and leave-one-out cross-validation technique on the subject-specific information to assess different epileptogenicity hypotheses regarding the location of pathological brain areas based on a priori knowledge from dynamical system properties. The Bayesian Virtual Epileptic Patient (BVEP) model, which relies on the fusion of structural data of individuals, a generative model of epileptiform discharges, and a self-tuning Monte Carlo sampling algorithm, is used to infer the spatial map of epileptogenicity across different brain areas. Our results indicate that measuring the out-of-sample prediction accuracy of the BVEP model with informative priors enables reliable and efficient evaluation of potential hypotheses regarding the degree of epileptogenicity across different brain regions. In contrast, while using uninformative priors, the information criteria are unable to provide strong evidence about the epileptogenicity of brain areas. We also show that the fully Bayesian criteria correctly assess different hypotheses about both structural and functional components of whole-brain models that differ across individuals. The fully Bayesian information-theory based approach used in this study suggests a patient-specific strategy for epileptogenicity hypothesis testing in generative brain network models of epilepsy to improve surgical outcomes.

Information criteria and cross validation for Bayesian inference in regular and singular cases

In data science, an unknown information source is estimated by a predictive distribution defined from a statistical model and a prior. In an older Bayesian framework, it was explained that the Bayesian predictive distribution should be the best on the assumption that a statistical model is convinced to be correct and a prior is given by a subjective belief in a small world. However, such a restricted treatment of Bayesian inference cannot be applied to highly complicated statistical models and learning machines in a large world. In 1980, a new scientific paradigm of Bayesian inference was proposed by Akaike, in which both a model and a prior are candidate systems and they had better be designed by mathematical procedures so that the predictive distribution is the better approximation of unknown information source. Nowadays, Akaike’s proposal is widely accepted in statistics, data science, and machine learning. In this paper, in order to establish a mathematical foundation for developing a measure of a statistical model and a prior, we show the relation among the generalization loss, the information criteria, and the cross-validation loss, then compare them from three different points of view. First, their performances are compared in singular problems where the posterior distribution is far from any normal distribution. Second, they are studied in the case when a leverage sample point is contained in data. And last, their stochastic properties are clarified when they are used for the prior optimization problem. The mathematical and experimental comparison shows the equivalence and the difference among them, which we expect useful in practical applications.

Bayesian splines versus fractional polynomials in network meta-analysis

Background

Network meta-analysis (NMA) provides a powerful tool for the simultaneous evaluation of multiple treatments by combining evidence from different studies, allowing for direct and indirect comparisons between treatments. In recent years, NMA is becoming increasingly popular in the medical literature and underlying statistical methodologies are evolving both in the frequentist and Bayesian framework. Traditional NMA models are often based on the comparison of two treatment arms per study. These individual studies may measure outcomes at multiple time points that are not necessarily homogeneous across studies.

Methods

In this article we present a Bayesian model based on B-splines for the simultaneous analysis of outcomes across time points, that allows for indirect comparison of treatments across different longitudinal studies.

Results

We illustrate the proposed approach in simulations as well as on real data examples available in the literature and compare it with a model based on P-splines and one based on fractional polynomials, showing that our approach is flexible and overcomes the limitations of the latter.

Conclusions

The proposed approach is computationally efficient and able to accommodate a large class of temporal treatment effect patterns, allowing for direct and indirect comparisons of widely varying shapes of longitudinal profiles.

Bayesian inference using Hamiltonian Monte-Carlo algorithm for nonlinear joint modeling in the context of cancer immunotherapy

Treatment evaluation in advanced cancer mainly relies on overall survival and tumor size dynamics. Both markers and their association can be simultaneously analyzed by using joint models, and these approaches are supported by many softwares or packages. However, these approaches are essentially limited to linear models for the longitudinal part, which limit their biological interpretation. More biological models of tumor dynamics can be obtained by using nonlinear models, but they are limited by the fact that parameter identifiability require rich dataset. In that context Bayesian approaches are particularly suited to incorporate the biological knowledge and increase the information available, but they are limited by the high computing cost of Monte-Carlo by Markov Chains algorithms. Here, we aimed to assess the performances of the Hamiltonian Monte-Carlo (HMC) algorithm implemented in Stan for inference in a nonlinear joint model. The method was validated on simulated data where HMC provided proper posterior distributions and credibility intervals in a reasonable computational time. Then the association between tumor size dynamics and survival was assessed in patients with advanced or metastatic bladder cancer treated with atezolizumab, an immunotherapy agent. HMC confirmed limited sensitivity to prior distributions. A cross-validation approach was developed and identified the current slope of tumor size dynamics as the most relevant driver of survival. In summary, HMC is an efficient approach to perform nonlinear joint models in a Bayesian framework, and opens the way for the use of nonlinear models to characterize both the rapid dynamics and the intersubject variability observed during cancer immunotherapy treatment.

Interindividual variability and lateralization of μ-opioid receptors in the human brain

Alterations in the brain's μ-opioid receptor (MOR) system have been associated with several neuropsychiatric disorders. Central MOR availability also varies considerably in healthy individuals. Multiple epidemiological factors have been proposed to influence the MOR system, but due to small sample sizes the magnitude of their influence remains inconclusive. We compiled [¹¹C]carfentanil positron emission tomography scans from 204 individuals with no neurologic or psychiatric disorders, and estimated the effects of sex, age, body mass index (BMI) and smoking on [¹¹C]carfentanil binding potential using between-subject regression analysis. We also examined hemispheric differences in MOR availability. Older age was associated with increase in MOR availability in frontotemporal areas but decrease in amygdala, thalamus, and nucleus accumbens. The age-dependent increase was stronger in males. MOR availability was globally lowered in smokers but independent of BMI. Finally, MOR availability was higher in the right versus the left hemisphere. The presently observed variation in MOR availability may explain why some individuals are prone to develop MOR-linked pathological states, such as chronic pain or psychiatric disorders. Lateralized MOR system may reflect hemispheric work specialization in central emotion and pain processes.

On the marginal likelihood and cross-validation

In Bayesian statistics, the marginal likelihood, also known as the evidence, is used to evaluate model fit as it quantifies the joint probability of the data under the prior. In contrast, non-Bayesian models are typically compared using cross-validation on held-out data, either through $k$-fold partitioning or leave-$p$-out subsampling. We show that the marginal likelihood is formally equivalent to exhaustive leave-$p$-out crossvalidation averaged over all values of $p$ and all held-out test sets when using the log posterior predictive probability as the scoring rule. Moreover, the log posterior predictive score is the only coherent scoring rule under data exchangeability. This offers new insight into the marginal likelihood and cross-validation, and highlights the potential sensitivity of the marginal likelihood to the choice of the prior. We suggest an alternative approach using cumulative cross-validation following a preparatory training phase. Our work has connections to prequential analysis and intrinsic Bayes factors, but is motivated in a different way.

The relative merit of empirical priors in non-identifiable and sloppy models: Applications to models of learning and decision-making : Empirical priors

Formal modeling approaches to cognition provide a principled characterization of observed responses in terms of a set of postulated processes, specifically in terms of parameters that modulate the latter. These model-based characterizations are useful to the extent that there is a clear, one-to-one mapping between parameters and model expectations (identifiability) and that parameters can be recovered from reasonably sized data using a typical experimental design (recoverability). These properties are sometimes not met for certain combinations of model classes and data. One suggestion to improve parameter identifiability and recoverability involves the use of "empirical priors", which constrain parameters according to a previously observed distribution of values. We assessed the efficacy of this proposal using a combination of real and artificial data. Our results showed that a point-estimate variant of the empirical-prior method could not improve parameter recovery systematically. We identified the source of poor parameter recovery in the low information content of the data. As a follow-up step, we developed a fully Bayesian variant of the empirical-prior method and assessed its performance. We find that even such a method that takes the covariance structure of the parameter distributions into account cannot reliably improve parameter recovery. We conclude that researchers should invest additional efforts in improving the informativeness of their experimental designs, as many of the problems associated to impoverished designs cannot be alleviated by modern statistical methods alone.

Bayesian residual analysis for spatially correlated data

This work considers residual analysis and predictive techniques for the identification of individual and multiple outliers in geostatistical data. The standardized Bayesian spatial residual is proposed and computed for three competing models: the Gaussian, Student-t and Gaussian-log-Gaussian spatial processes. In this context, the spatial models are investigated regarding their plausibility for datasets contaminated with outliers. The posterior probability of an outlying observation is computed based on the standardized residuals and different thresholds for outlier discrimination are tested. From a predictive point of view, methods such as the conditional predictive ordinate, the predictive concordance and the Savage–Dickey density ratio for hypothesis testing are investigated for identification of outliers in the spatial setting. For illustration, contaminated datasets are considered to assess the performance of the three spatial models for identification of outliers in spatial data. Furthermore, an application to wind speed modelling is presented to illustrate the usefulness of the proposed tools to detect regions with large wind speeds.

Improving genomics-based predictions for precision medicine through active elicitation of expert knowledge

Motivation

Precision medicine requires the ability to predict the efficacies of different treatments for a given individual using high-dimensional genomic measurements. However, identifying predictive features remains a challenge when the sample size is small. Incorporating expert knowledge offers a promising approach to improve predictions, but collecting such knowledge is laborious if the number of candidate features is very large.

Results

We introduce a probabilistic framework to incorporate expert feedback about the impact of genomic measurements on the outcome of interest and present a novel approach to collect the feedback efficiently, based on Bayesian experimental design. The new approach outperformed other recent alternatives in two medical applications: prediction of metabolic traits and prediction of sensitivity of cancer cells to different drugs, both using genomic features as predictors. Furthermore, the intelligent approach to collect feedback reduced the workload of the expert to approximately 11%, compared to a baseline approach.

Availability and implementation

Source code implementing the introduced computational methods is freely available at https://github.com/AaltoPML/knowledge-elicitation-for-precision-medicine.

Supplementary information

Supplementary data are available at Bioinformatics online.

Florida Populations Most at Risk of Not Being Up to Date With Colorectal Cancer Screening

INTRODUCTION

The purpose of this study was to examine the characteristics of populations at risk of not being up to date on colorectal cancer screening in Florida.

METHODS

We used Exhaustive Chi-squared Automatic Interaction Detection, a classification tree analysis, to identify subgroups not up to date with colorectal cancer screening using the 2013 Florida Behavioral Risk Factor Surveillance System. The data set was restricted to adults aged 50 to 75 years (n = 14,756).

RESULTS

Only 65.5% of the sample was up to date on colorectal cancer screening. Having no insurance and having a primary care provider were the most significant predictors of not being up to date on screening. The highest risk subgroups were 1) respondents with no insurance and no primary care provider, regardless of their employment status (screening rate, 12.1%-23.7%); 2) respondents with no insurance but had a primary care provider and were employed (screening rate, 32.3%); and 3) respondents with insurance, who were younger than 55 years, and who were current smokers (screening rate, 42.0%).

CONCLUSION

Some populations in Florida are at high risk for not being up to date on colorectal cancer screening. To achieve Healthy People 2020 goals, interventions may need to be further tailored to target these subgroups.

A Bayesian Spatial Model to Predict Disease Status Using Imaging Data From Various Modalities

Relating disease status to imaging data stands to increase the clinical significance of neuroimaging studies. Many neurological and psychiatric disorders involve complex, systems-level alterations that manifest in functional and structural properties of the brain and possibly other clinical and biologic measures. We propose a Bayesian hierarchical model to predict disease status, which is able to incorporate information from both functional and structural brain imaging scans. We consider a two-stage whole brain parcellation, partitioning the brain into 282 subregions, and our model accounts for correlations between voxels from different brain regions defined by the parcellations. Our approach models the imaging data and uses posterior predictive probabilities to perform prediction. The estimates of our model parameters are based on samples drawn from the joint posterior distribution using Markov Chain Monte Carlo (MCMC) methods. We evaluate our method by examining the prediction accuracy rates based on leave-one-out cross validation, and we employ an importance sampling strategy to reduce the computation time. We conduct both whole-brain and voxel-level prediction and identify the brain regions that are highly associated with the disease based on the voxel-level prediction results. We apply our model to multimodal brain imaging data from a study of Parkinson's disease. We achieve extremely high accuracy, in general, and our model identifies key regions contributing to accurate prediction including caudate, putamen, and fusiform gyrus as well as several sensory system regions.

Bayesian Multiresolution Variable Selection for Ultra-High Dimensional Neuroimaging Data

Ultra-high dimensional variable selection has become increasingly important in analysis of neuroimaging data. For example, in the Autism Brain Imaging Data Exchange (ABIDE) study, neuroscientists are interested in identifying important biomarkers for early detection of the autism spectrum disorder (ASD) using high resolution brain images that include hundreds of thousands voxels. However, most existing methods are not feasible for solving this problem due to their extensive computational costs. In this work, we propose a novel multiresolution variable selection procedure under a Bayesian probit regression framework. It recursively uses posterior samples for coarser-scale variable selection to guide the posterior inference on finer-scale variable selection, leading to very efficient Markov chain Monte Carlo (MCMC) algorithms. The proposed algorithms are computationally feasible for ultra-high dimensional data. Also, our model incorporates two levels of structural information into variable selection using Ising priors: the spatial dependence between voxels and the functional connectivity between anatomical brain regions. Applied to the resting state functional magnetic resonance imaging (R-fMRI) data in the ABIDE study, our methods identify voxel-level imaging biomarkers highly predictive of the ASD, which are biologically meaningful and interpretable. Extensive simulations also show that our methods achieve better performance in variable selection compared to existing methods.

Estimating cross-validatory predictive p-values with integrated importance sampling for disease mapping models

An important statistical task in disease mapping problems is to identify divergent regions with unusually high or low risk of disease. Leave-one-out cross-validatory (LOOCV) model assessment is the gold standard for estimating predictive p-values that can flag such divergent regions. However, actual LOOCV is time-consuming because one needs to rerun a Markov chain Monte Carlo analysis for each posterior distribution in which an observation is held out as a test case. This paper introduces a new method, called integrated importance sampling (iIS), for estimating LOOCV predictive p-values with only Markov chain samples drawn from the posterior based on a full data set. The key step in iIS is that we integrate away the latent variables associated the test observation with respect to their conditional distribution without reference to the actual observation. By following the general theory for importance sampling, the formula used by iIS can be proved to be equivalent to the LOOCV predictive p-value. We compare iIS and other three existing methods in the literature with two disease mapping datasets. Our empirical results show that the predictive p-values estimated with iIS are almost identical to the predictive p-values estimated with actual LOOCV and outperform those given by the existing three methods, namely, the posterior predictive checking, the ordinary importance sampling, and the ghosting method by Marshall and Spiegelhalter (2003). Copyright © 2017 John Wiley & Sons, Ltd.

Cross-Validation Without Doing Cross-Validation in Genome-Enabled Prediction

Cross-validation of methods is an essential component of genome-enabled prediction of complex traits. We develop formulae for computing the predictions that would be obtained when one or several cases are removed in the training process, to become members of testing sets, but by running the model using all observations only once. Prediction methods to which the developments apply include least squares, best linear unbiased prediction (BLUP) of markers, or genomic BLUP, reproducing kernels Hilbert spaces regression with single or multiple kernel matrices, and any member of a suite of linear regression methods known as “Bayesian alphabet.” The approach used for Bayesian models is based on importance sampling of posterior draws. Proof of concept is provided by applying the formulae to a wheat data set representing 599 inbred lines genotyped for 1279 markers, and the target trait was grain yield. The data set was used to evaluate predictive mean-squared error, impact of alternative layouts on maximum likelihood estimates of regularization parameters, model complexity, and residual degrees of freedom stemming from various strengths of regularization, as well as two forms of importance sampling. Our results will facilitate carrying out extensive cross-validation without model retraining for most machines employed in genome-assisted prediction of quantitative traits.

A novel method for expediting the development of patient-reported outcome measures and an evaluation across several populations

Item response theory (IRT) models provide an appropriate alternative to the classical ordinal confirmatory factor analysis (CFA) during the development of patient-reported outcome measures (PROMs). Current literature has identified the assessment of IRT model fit as both challenging and underdeveloped (Sinharay & Johnson, 2003; Sinharay, Johnson, & Stern, 2006). This study evaluates the performance of Ordinal Bayesian Instrument Development (OBID), a Bayesian IRT model with a probit link function approach, through applications in two breast cancer-related instrument development studies. The primary focus is to investigate an appropriate method for comparing Bayesian IRT models in PROMs development. An exact Bayesian leave-one-out cross-validation (LOO-CV) approach (Vehtari & Lampinen, 2002) is implemented to assess prior selection for the item discrimination parameter in the IRT model and subject content experts' bias (in a statistical sense and not to be confused with psychometric bias as in differential item functioning) toward the estimation of item-to-domain correlations. Results support the utilization of content subject experts' information in establishing evidence for construct validity when sample size is small. However, the incorporation of subject experts' content information in the OBID approach can be sensitive to the level of expertise of the recruited experts. More stringent efforts need to be invested in the appropriate selection of subject experts to efficiently use the OBID approach and reduce potential bias during PROMs development.

Probabilistic Inference: Task Dependency and Individual Differences of Probability Weighting Revealed by Hierarchical Bayesian Modeling

Cognitive determinants of probabilistic inference were examined using hierarchical Bayesian modeling techniques. A classic urn-ball paradigm served as experimental strategy, involving a factorial two (prior probabilities) by two (likelihoods) design. Five computational models of cognitive processes were compared with the observed behavior. Parameter-free Bayesian posterior probabilities and parameter-free base rate neglect provided inadequate models of probabilistic inference. The introduction of distorted subjective probabilities yielded more robust and generalizable results. A general class of (inverted) S-shaped probability weighting functions had been proposed; however, the possibility of large differences in probability distortions not only across experimental conditions, but also across individuals, seems critical for the model's success. It also seems advantageous to consider individual differences in parameters of probability weighting as being sampled from weakly informative prior distributions of individual parameter values. Thus, the results from hierarchical Bayesian modeling converge with previous results in revealing that probability weighting parameters show considerable task dependency and individual differences. Methodologically, this work exemplifies the usefulness of hierarchical Bayesian modeling techniques for cognitive psychology. Theoretically, human probabilistic inference might be best described as the application of individualized strategic policies for Bayesian belief revision.

Involvement of Sensory Regions in Affective Experience: A Meta-Analysis

A growing body of work suggests that sensory processes may also contribute to affective experience. In this study, we performed a meta-analysis of affective experiences driven through visual, auditory, olfactory, gustatory, and somatosensory stimulus modalities including study contrasts that compared affective stimuli to matched neutral control stimuli. We found, first, that limbic and paralimbic regions, including the amygdala, anterior insula, pre-supplementary motor area, and portions of orbitofrontal cortex were consistently engaged across two or more modalities. Second, early sensory input regions in occipital, temporal, piriform, mid-insular, and primary sensory cortex were frequently engaged during affective experiences driven by visual, auditory, olfactory, gustatory, and somatosensory inputs. A classification analysis demonstrated that the pattern of neural activity across a contrast map diagnosed the stimulus modality driving the affective experience. These findings suggest that affective experiences are constructed from activity that is distributed across limbic and paralimbic brain regions and also activity in sensory cortical regions.

The influence of selective participation in a physical activity intervention on the generalizability of findings

OBJECTIVE

To evaluate factors that characterize employees who did not participate in a physical activity intervention in an occupational setting and assess how selective participation affects inferences from the data.

METHODS

Employees were asked to complete a health risk appraisal. The respondents were invited to participate in a physical activity intervention. We compared predictors of sickness absence (register data) among all respondents and those who participated in the intervention, using Bayesian regression analysis.

RESULTS

Of 1116 employees, 817 (73%) responded, of whom 544 (67%) participated in the intervention. Participants had better health behaviors and fewer health problems than those who did not participate. The predictors of sickness absence in all respondents differed from those who participated in the intervention.

CONCLUSIONS

Selective participation may reduce the potential benefit of interventions and limit generalizability of findings.

Development of Vermont assessment of sex offender risk-2 (VASOR-2) reoffense risk scale

The present study aimed to revise the Vermont Assessment of Sex Offender Risk (VASOR) Reoffense Risk Scale, a commonly used sex offender risk assessment tool. The revised tool was named the VASOR-2. Among models tested to revise the scale, a logistic regression model showed the best balance between simplicity of use, goodness of fit, and internal validity (as tested with K-10 cross-validation), and maximized predictive accuracy. Predictive accuracy was tested using four meta-analytically combined data sets drawn from Canada and Vermont (N = 1,581). At 5-year fixed follow-up, the predictive accuracy for sexual recidivism for VASOR-2 (AUC = .74) was similar to the VASOR (AUC = .71). The findings show the VASOR-2 is well calibrated with observed recidivism rates for all but the highest risk sex offenders. The instrument showed good interrater reliability (ICC = .88). An advantage of the VASOR-2 is that it has fewer items and simpler scoring instructions than the VASOR. Norms are presented for a contemporary, nonselected, routine sample of Vermont sex offenders (n = 887).

A multilevel model for spatially correlated binary data in the presence of misclassification: an application in oral health research

Dental caries is a highly prevalent disease affecting the tooth's hard tissues by acid-forming bacteria. The past and present caries status of a tooth is characterized by a response called caries experience (CE). Several epidemiological studies have explored risk factors for CE. However, the detection of CE is prone to misclassification because some cases are neither clearly carious nor noncarious, and this needs to be incorporated into the epidemiological models for CE data. From a dentist's point of view, it is most appealing to analyze CE on the tooth's surface, implying that the multilevel structure of the data (surface-tooth-mouth) needs to be taken into account. In addition, CE data are spatially referenced, that is, an active lesion on one surface may impact the decay process of the neighboring surfaces, and that might also influence the process of scoring CE. In this paper, we investigate two hypotheses: that is, (i) CE outcomes recorded at surface level are spatially associated; and (ii) the dental examiners exhibit some spatial behavior while scoring CE at surface level, by using a spatially referenced multilevel autologistic model, corrected for misclassification. These hypotheses were tested on the well-known Signal Tandmobiel® study on dental caries, and simulation studies were conducted to assess the effect of misclassification and strength of spatial dependence on the autologistic model parameters. Our results indicate a substantial spatial dependency in the examiners' scoring behavior and also in the prevalence of CE at surface level.

Optimized atomic statistical potentials: assessment of protein interfaces and loops

MOTIVATION

Statistical potentials have been widely used for modeling whole proteins and their parts (e.g. sidechains and loops) as well as interactions between proteins, nucleic acids and small molecules. Here, we formulate the statistical potentials entirely within a statistical framework, avoiding questionable statistical mechanical assumptions and approximations, including a definition of the reference state.

RESULTS

We derive a general Bayesian framework for inferring statistically optimized atomic potentials (SOAP) in which the reference state is replaced with data-driven 'recovery' functions. Moreover, we restrain the relative orientation between two covalent bonds instead of a simple distance between two atoms, in an effort to capture orientation-dependent interactions such as hydrogen bonds. To demonstrate this general approach, we computed statistical potentials for protein-protein docking (SOAP-PP) and loop modeling (SOAP-Loop). For docking, a near-native model is within the top 10 scoring models in 40% of the PatchDock benchmark cases, compared with 23 and 27% for the state-of-the-art ZDOCK and FireDock scoring functions, respectively. Similarly, for modeling 12-residue loops in the PLOP benchmark, the average main-chain root mean square deviation of the best scored conformations by SOAP-Loop is 1.5 Å, close to the average root mean square deviation of the best sampled conformations (1.2 Å) and significantly better than that selected by Rosetta (2.1 Å), DFIRE (2.3 Å), DOPE (2.5 Å) and PLOP scoring functions (3.0 Å). Our Bayesian framework may also result in more accurate statistical potentials for additional modeling applications, thus affording better leverage of the experimentally determined protein structures.

AVAILABILITY AND IMPLEMENTATION

SOAP-PP and SOAP-Loop are available as part of MODELLER (http://salilab.org/modeller).

Gastrointestinal stromal tumor: a method for optimizing the timing of CT scans in the follow-up of cancer patients

PURPOSE

To develop a mathematical model to adjust the timing of computed tomography (CT) scans with the hazard of cancer recurrence in time to facilitate early detection of cancer recurrence.

MATERIALS AND METHODS

The clinical data were extracted from the randomized Scandinavian Sarcoma Group (SSG) XVIII/Arbeitsgemeinschaft Internistische Onkologie (AIO) trial database. The SSG XVIII/AIO trial was registered (trial no. NCT00116935) and approved by the national or institutional review boards. In the trial, 1- and 3-year durations of adjuvant imatinib mesylate in the treatment of patients with gastrointestinal stromal tumor (GIST) were compared. A nonhomogeneous Poisson model with a piecewise log-constant hazard in time that accounts for the nonlinear pattern of GIST recurrence was applied to tumor site, mitotic count, and recurrence data. The optimal times to obtain follow-up CT scans were computed by modifying the frequency of CT scans with the hazard of tumor recurrence in time. The hazard-adjusted follow-up schedules were compared with the National Comprehensive Cancer Network (NCCN) guidelines of the United States, which suggest imaging with CT at intervals of 3-6 months for 3-5 years and then annually.

RESULTS

Optimized timing of CT scans on the basis of hazard of recurrence resulted in follow-up schedule options where CT is performed more sparsely than in the NCCN guidelines during adjuvant imatinib administration and more frequently, at approximately 3-month intervals, during the first 2 years that follow imatinib discontinuation when the risk of recurrence was the greatest. The number of CT scans could be reduced by a median of 31% (from 13 to nine) compared with the standard schedules within the first 6 years of follow-up without increasing the delay in recurrence detection.

CONCLUSION

Detection of GIST recurrence may be enhanced by adjusting the timing of the CT scans with the hazard of recurrence. The method may be applicable to other human tumor types. Online supplemental material is available for this article.

Approximate Bayesian computation

Approximate Bayesian computation (ABC) constitutes a class of computational methods rooted in Bayesian statistics. In all model-based statistical inference, the likelihood function is of central importance, since it expresses the probability of the observed data under a particular statistical model, and thus quantifies the support data lend to particular values of parameters and to choices among different models. For simple models, an analytical formula for the likelihood function can typically be derived. However, for more complex models, an analytical formula might be elusive or the likelihood function might be computationally very costly to evaluate. ABC methods bypass the evaluation of the likelihood function. In this way, ABC methods widen the realm of models for which statistical inference can be considered. ABC methods are mathematically well-founded, but they inevitably make assumptions and approximations whose impact needs to be carefully assessed. Furthermore, the wider application domain of ABC exacerbates the challenges of parameter estimation and model selection. ABC has rapidly gained popularity over the last years and in particular for the analysis of complex problems arising in biological sciences (e.g., in population genetics, ecology, epidemiology, and systems biology).

Bayesian population approaches to the analysis of dose escalation studies

In dose escalation studies cohorts of subjects are given increasing doses of a candidate drug to assess safety and tolerability, pharmacokinetics and pharmacological response. The escalation is carried on until a predefined stopping limit is achieved, often identified by a pharmacokinetic endpoint such as peak plasma concentration or area under the plasma concentration-time profile. In the present work, the application of Bayesian methodologies to Phase I dose escalation studies is explored. A Bayesian population model is devised, which provides predictions of dose-response and dose-risk curves, both for individuals already enrolled in the trial and for a new, previously untested subject. Empirical and fully Bayesian estimation algorithms are worked out. Such methods provide equivalent performances on both experimental and simulated datasets. With respect to previous work, it is quantitatively proven not only that a more general and flexible model is identifiable, but also that such flexibility is needed in real scenarios.

Risk of recurrence of gastrointestinal stromal tumour after surgery: an analysis of pooled population-based cohorts

BACKGROUND

The risk of recurrence of gastrointestinal stromal tumour (GIST) after surgery needs to be estimated when considering adjuvant systemic therapy. We assessed prognostic factors of patients with operable GIST, to compare widely used risk-stratification schemes and to develop a new method for risk estimation.

METHODS

Population-based cohorts of patients diagnosed with operable GIST, who were not given adjuvant therapy, were identified from the literature. Data from ten series and 2560 patients were pooled. Risk of tumour recurrence was stratified using the National Institute of Health (NIH) consensus criteria, the modified consensus criteria, and the Armed Forces Institute of Pathology (AFIP) criteria. Prognostic factors were examined using proportional hazards and non-linear models. The results were validated in an independent centre-based cohort consisting of 920 patients with GIST.

FINDINGS

Estimated 15-year recurrence-free survival (RFS) after surgery was 59·9% (95% CI 56·2-63·6); few recurrences occurred after the first 10 years of follow-up. Large tumour size, high mitosis count, non-gastric location, presence of rupture, and male sex were independent adverse prognostic factors. In receiver operating characteristics curve analysis of 10-year RFS, the NIH consensus criteria, modified consensus criteria, and AFIP criteria resulted in an area under the curve (AUC) of 0·79 (95% CI 0·76-0·81), 0·78 (0·75-0·80), and 0·82 (0·80-0·85), respectively. The modified consensus criteria identified a single high-risk group. Since tumour size and mitosis count had a non-linear association with the risk of GIST recurrence, novel prognostic contour maps were generated using non-linear modelling of tumour size and mitosis count, and taking into account tumour site and rupture. The non-linear model accurately predicted the risk of recurrence (AUC 0·88, 0·86-0·90).

INTERPRETATION

The risk-stratification schemes assessed identify patients who are likely to be cured by surgery alone. Although the modified NIH classification is the best criteria to identify a single high-risk group for consideration of adjuvant therapy, the prognostic contour maps resulting from non-linear modelling are appropriate for estimation of individualised outcomes.

FUNDING

Academy of Finland, Cancer Society of Finland, Sigrid Juselius Foundation and Helsinki University Research Funds.

Bayesian population modeling of phase I dose escalation studies: Gaussian process versus parametric approaches

The early stages of the drug development process are often characterized by a limited number of subjects participating the study and a limited number of measurements per individual that can be collected, mainly due to technical, ethical, and cost reasons. The so-called dose escalation studies, performed during phase I, usually involve about 40 subjects or less, and feature observations at no more than three (rarely four or five) dose levels-per-subject. Depending on the complexity of the underlying pharmacokinetics, simple linear models or nonlinear ones (e.g., power, E(max) models) may be appropriate to describe the relationship between the metrics of systemic exposure to the drug (C(max), AUC) and the administered dose. However, in such data-poor scenarios, formulating models based on parametric descriptions is generally hard, and may easily result in model misspecification. Hence, nonparametric or "model-free" solutions, borrowed from the machine learning field, are deemed appealing. We resort to Gaussian process theory to work out Bayesian posterior expectations of a population (a.k.a mixed-effects) regression problem, namely Population Smoothing Splines (PSS). We show that in seven experimental dose escalation studies, Population Smoothing Splines improve on three widely used parametric population methods. Superiority of the model-free technique is confirmed by a simulated benchmark: Population Smoothing Splines compare very favorably even with the true parametric model structure underlying the simulated data.

Approximate inference for disease mapping with sparse Gaussian processes

Gaussian process (GP) models are widely used in disease mapping as they provide a natural framework for modeling spatial correlations. Their challenges, however, lie in computational burden and memory requirements. In disease mapping models, the other difficulty is inference, which is analytically intractable due to the non-Gaussian observation model. In this paper, we address both these challenges. We show how to efficiently build fully and partially independent conditional (FIC/PIC) sparse approximations for the GP in two-dimensional surface, and how to conduct approximate inference using expectation propagation (EP) algorithm and Laplace approximation (LA). We also propose to combine FIC with a compactly supported covariance function to construct a computationally efficient additive model that can model long and short length-scale spatial correlations simultaneously. The benefit of these approximations is computational. The sparse GPs speed up the computations and reduce the memory requirements. The posterior inference via EP and Laplace approximation is much faster and is practically as accurate as via Markov chain Monte Carlo.

An application of change-point recursive models to the relationship between litter size and number of stillborns in pigs

We developed and implemented change-point recursive models and compared them with a linear recursive model and a standard mixed model (SMM), in the scope of the relationship between litter size (LS) and number of stillborns (NSB) in pigs. The proposed approach allows us to estimate the point of change in multiple-segment modeling of a nonlinear relationship between phenotypes. We applied the procedure to a data set provided by a commercial Large White selection nucleus. The data file consisted of LS and NSB records of 4,462 parities. The results of the analysis clearly identified the location of the change points between different structural regression coefficients. The magnitude of these coefficients increased with LS, indicating an increasing incidence of LS on the NSB ratio. However, posterior distributions of correlations were similar across subpopulations (defined by the change points on LS), except for those between residuals. The heritability estimates of NSB did not present differences between recursive models. Nevertheless, these heritabilities were greater than those obtained for SMM (0.05) with a posterior probability of 85%. These results suggest a nonlinear relationship between LS and NSB, which supports the adequacy of a change-point recursive model for its analysis. Furthermore, the results from model comparisons support the use of recursive models. However, the adequacy of the different recursive models depended on the criteria used: the linear recursive model was preferred on account of its smallest deviance value, whereas nonlinear recursive models provided a better fit and predictive ability based on the cross-validation approach.