A new type of generalized information criterion for regularization parameter selection in penalized regression with application to treatment process data

We propose a new approach to select the regularization parameter using a new version of the generalized information criterion (GIC ) in the subject of penalized regression. We prove the identifiability of bridge regression model as a prerequisite of statistical modeling. Then, we propose asymptotically efficient generalized information criterion (AGIC ) and prove that it has asymptotic loss efficiency. Also, we verified the better performance of AGIC in comparison to the older versions of GIC . Furthermore, we propose MSE search paths to order the selected features by lasso regression based on numerical studies. The MSE search paths provide a way to cover the lack of feature ordering in lasso regression model. The performance of AGIC with other types of GIC is compared using MSE and model utility in simulation study. We exert AGIC and other criteria to analyze breast and prostate cancer and Parkinson disease datasets. The results confirm the superiority of AGIC in almost all situations.

A Bayesian Measure of Model Accuracy

Ensuring that the proposed probabilistic model accurately represents the problem is a critical step in statistical modeling, as choosing a poorly fitting model can have significant repercussions on the decision-making process. The primary objective of statistical modeling often revolves around predicting new observations, highlighting the importance of assessing the model's accuracy. However, current methods for evaluating predictive ability typically involve model comparison, which may not guarantee a good model selection. This work presents an accuracy measure designed for evaluating a model's predictive capability. This measure, which is straightforward and easy to understand, includes a decision criterion for model rejection. The development of this proposal adopts a Bayesian perspective of inference, elucidating the underlying concepts and outlining the necessary procedures for application. To illustrate its utility, the proposed methodology was applied to real-world data, facilitating an assessment of its practicality in real-world scenarios.

A high internal heat flux and large core in a warm Neptune exoplanet

Interactions between exoplanetary atmospheres and internal properties have long been proposed to be drivers of the inflation mechanisms of gaseous planets and apparent atmospheric chemical disequilibrium conditions1. However, transmission spectra of exoplanets have been limited in their ability to observationally confirm these theories owing to the limited wavelength coverage of the Hubble Space Telescope (HST) and inferences of single molecules, mostly H2O (ref. 2). In this work, we present the panchromatic transmission spectrum of the approximately 750 K, low-density, Neptune-sized exoplanet WASP-107b using a combination of HST Wide Field Camera 3 (WFC3) and JWST Near-Infrared Camera (NIRCam) and Mid-Infrared Instrument (MIRI). From this spectrum, we detect spectroscopic features resulting from H2O (21σ), CH4 (5σ), CO (7σ), CO2 (29σ), SO2 (9σ) and NH3 (6σ). The presence of these molecules enables constraints on the atmospheric metal enrichment (M/H is 10-18× solar3), vertical mixing strength (log10Kzz = 8.4-9.0 cm2 s-1) and internal temperature (>345 K). The high internal temperature is suggestive of tidally driven inflation4 acting on a Neptune-like internal structure, which can naturally explain the large radius and low density of the planet. These findings suggest that eccentricity-driven tidal heating is a critical process governing atmospheric chemistry and interior-structure inferences for most of the cool (<1,000 K) super-Earth-to-Saturn-mass exoplanet population.

Improved RNA stability estimation through Bayesian modeling reveals most Salmonella transcripts have subminute half-lives

RNA decay is a crucial mechanism for regulating gene expression in response to environmental stresses. In bacteria, RNA-binding proteins (RBPs) are known to be involved in posttranscriptional regulation, but their global impact on RNA half-lives has not been extensively studied. To shed light on the role of the major RBPs ProQ and CspC/E in maintaining RNA stability, we performed RNA sequencing of Salmonella enterica over a time course following treatment with the transcription initiation inhibitor rifampicin (RIF-seq) in the presence and absence of these RBPs. We developed a hierarchical Bayesian model that corrects for confounding factors in rifampicin RNA stability assays and enables us to identify differentially decaying transcripts transcriptome-wide. Our analysis revealed that the median RNA half-life in Salmonella in early stationary phase is less than 1 min, a third of previous estimates. We found that over half of the 500 most long-lived transcripts are bound by at least one major RBP, suggesting a general role for RBPs in shaping the transcriptome. Integrating differential stability estimates with cross-linking and immunoprecipitation followed by RNA sequencing (CLIP-seq) revealed that approximately 30% of transcripts with ProQ binding sites and more than 40% with CspC/E binding sites in coding or 3' untranslated regions decay differentially in the absence of the respective RBP. Analysis of differentially destabilized transcripts identified a role for ProQ in the oxidative stress response. Our findings provide insights into posttranscriptional regulation by ProQ and CspC/E, and the importance of RBPs in regulating gene expression.

Innovative approaches in soil carbon sequestration modelling for better prediction with limited data

Soil carbon accounting and prediction play a key role in building decision support systems for land managers selling carbon credits, in the spirit of the Paris and Kyoto protocol agreements. Land managers typically rely on computationally complex models fit using sparse datasets to make these accounts and predictions. The model complexity and sparsity of the data can lead to over-fitting, leading to inaccurate results when making predictions with new data. Modellers address over-fitting by simplifying their models and reducing the number of parameters, and in the current context this could involve neglecting some soil organic carbon (SOC) components. In this study, we introduce two novel SOC models and a new RothC-like model and investigate how the SOC components and complexity of the SOC models affect the SOC prediction in the presence of small and sparse time series data. We develop model selection methods that can identify the soil carbon model with the best predictive performance, in light of the available data. Through this analysis we reveal that commonly used complex soil carbon models can over-fit in the presence of sparse time series data, and our simpler models can produce more accurate predictions.

Bayesian hierarchical models incorporating study-level covariates for multivariate meta-analysis of diagnostic tests without a gold standard with application to COVID-19

When evaluating a diagnostic test, it is common that a gold standard may not be available. One example is the diagnosis of SARS-CoV-2 infection using saliva sampling or nasopharyngeal swabs. Without a gold standard, a pragmatic approach is to postulate a "reference standard," defined as positive if either test is positive, or negative if both are negative. However, this pragmatic approach may overestimate sensitivities because subjects infected with SARS-CoV-2 may still have double-negative test results even when both tests exhibit perfect specificity. To address this limitation, we propose a Bayesian hierarchical model for simultaneously estimating sensitivity, specificity, and disease prevalence in the absence of a gold standard. The proposed model allows adjusting for study-level covariates. We evaluate the model performance using an example based on a recently published meta-analysis on the diagnosis of SARS-CoV-2 infection and extensive simulations. Compared with the pragmatic reference standard approach, we demonstrate that the proposed Bayesian method provides a more accurate evaluation of prevalence, specificity, and sensitivity in a meta-analytic framework.

Novel Information-Driven Smoothing Spline Linearization Method for High-Precision Displacement Sensors Based on Information Criterions

A noise-resistant linearization model that reveals the true nonlinearity of the sensor is essential for retrieving accurate physical displacement from the signals captured by sensing electronics. In this paper, we propose a novel information-driven smoothing spline linearization method, which innovatively integrates one new and three standard information criterions into a smoothing spline for the high-precision displacement sensors' linearization. Using theoretical analysis and Monte Carlo simulation, the proposed linearization method is demonstrated to outperform traditional polynomial and spline linearization methods for high-precision displacement sensors with a low noise to range ratio in the 10-5 level. Validation experiments were carried out on two different types of displacement sensors to benchmark the performance of the proposed method compared to the polynomial models and the the non-smoothing cubic spline. The results show that the proposed method with the new modified Akaike Information Criterion stands out compared to the other linearization methods and can improve the residual nonlinearity by over 50% compared to the standard polynomial model. After being linearized via the proposed method, the residual nonlinearities reach as low as ±0.0311% F.S. (Full Scale of Range), for the 1.5 mm range chromatic confocal displacement sensor, and ±0.0047% F.S., for the 100 mm range laser triangulation displacement sensor.

Filial Cannibalism Leads to Chronic Nest Failure of Eastern Hellbender Salamanders (Cryptobranchus alleganiensis)

AbstractIn species that provide parental care, parents will sometimes cannibalize their own young (i.e., filial cannibalism). Here, we quantified the frequency of whole-clutch filial cannibalism in a species of giant salamander (eastern hellbender; Cryptobranchus alleganiensis) that has experienced precipitous population declines with unknown causes. We used underwater artificial nesting shelters deployed across a gradient of upstream forest cover to assess the fates of 182 nests at 10 sites over 8 years. We found strong evidence that nest failure rates increased at sites with low riparian forest cover in the upstream catchment. At several sites, reproductive failure was 100%, mainly due to cannibalism by the caring male. The high incidence of filial cannibalism at degraded sites was not explained by evolutionary hypotheses for filial cannibalism based on poor adult body condition or low reproductive value of small clutches. Instead, larger clutches at degraded sites were most vulnerable to cannibalism. We hypothesize that high frequencies of filial cannibalism of large clutches in areas with low forest cover could be related to changes in water chemistry or siltation that influence parental physiology or that reduce the viability of eggs. Importantly, our results identify chronic nest failure as a possible mechanism contributing to population declines and observed geriatric age structure in this imperiled species.

On predictive inference for intractable models via approximate Bayesian computation

Approximate Bayesian computation (ABC) is commonly used for parameter estimation and model comparison for intractable simulator-based statistical models whose likelihood function cannot be evaluated. In this paper we instead investigate the feasibility of ABC as a generic approximate method for predictive inference, in particular, for computing the posterior predictive distribution of future observations or missing data of interest. We consider three complementary ABC approaches for this goal, each based on different assumptions regarding which predictive density of the intractable model can be sampled from. The case where only simulation from the joint density of the observed and future data given the model parameters can be used for inference is given particular attention and it is shown that the ideal summary statistic in this setting is minimal predictive sufficient instead of merely minimal sufficient (in the ordinary sense). An ABC prediction approach that takes advantage of a certain latent variable representation is also investigated. We additionally show how common ABC sampling algorithms can be used in the predictive settings considered. Our main results are first illustrated by using simple time-series models that facilitate analytical treatment, and later by using two common intractable dynamic models.

SUPPLEMENTARY INFORMATION

The online version contains supplementary material available at 10.1007/s11222-022-10163-6.

A dynamic causal modeling of the second outbreak of COVID-19 in Italy

While the vaccination campaign against COVID-19 is having its positive impact, we retrospectively analyze the causal impact of some decisions made by the Italian government on the second outbreak of the SARS-CoV-2 pandemic in Italy, when no vaccine was available. First, we analyze the causal impact of reopenings after the first lockdown in 2020. In addition, we also analyze the impact of reopening schools in September 2020. Our results provide an unprecedented opportunity to evaluate the causal relationship between the relaxation of restrictions and the transmission in the community of a highly contagious respiratory virus that causes severe illness in the absence of prophylactic vaccination programs. We present a purely data-analytic approach based on a Bayesian methodology and discuss possible interpretations of the results obtained and implications for policy makers.

A monotone single index model for missing-at-random longitudinal proportion data

Beta distributions are commonly used to model proportion valued response variables, often encountered in longitudinal studies. In this article, we develop semi-parametric Beta regression models for proportion valued responses, where the aggregate covariate effect is summarized and flexibly modeled, using a interpretable monotone time-varying single index transform of a linear combination of the potential covariates. We utilize the potential of single index models, which are effective dimension reduction tools and accommodate link function misspecification in generalized linear mixed models. Our Bayesian methodology incorporates the missing-at-random feature of the proportion response and utilize Hamiltonian Monte Carlo sampling to conduct inference. We explore finite-sample frequentist properties of our estimates and assess the robustness via detailed simulation studies. Finally, we illustrate our methodology via application to a motivating longitudinal dataset on obesity research recording proportion body fat.

How to Choose between Different Bayesian Posterior Indices for Hypothesis Testing in Practice

Hypothesis testing is an essential statistical method in experimental psychology and the cognitive sciences. The problems of traditional null hypothesis significance testing (NHST) have been discussed widely, and among the proposed solutions to the replication problems caused by the inappropriate use of significance tests and p-values is a shift toward Bayesian data analysis. However, Bayesian hypothesis testing is concerned with various posterior indices for significance and the size of an effect. This complicates Bayesian hypothesis testing in practice, as the availability of multiple Bayesian alternatives to the traditional p-value causes confusion which one to select and why. In this paper, various Bayesian posterior indices which have been proposed in the literature are compared and their benefits and limitations are discussed. The comparison shows that conceptually not all proposed Bayesian alternatives to NHST and p-values are beneficial, and the usefulness of some indices strongly depends on the study design and research goal. However, the comparison also reveals that there exist at least two candidates among the available Bayesian posterior indices which have appealing theoretical properties and are widely underused in the cognitive sciences.

Using reference models in variable selection

Variable selection, or more generally, model reduction is an important aspect of the statistical workflow aiming to provide insights from data. In this paper, we discuss and demonstrate the benefits of using a reference model in variable selection. A reference model acts as a noise-filter on the target variable by modeling its data generating mechanism. As a result, using the reference model predictions in the model selection procedure reduces the variability and improves stability, leading to improved model selection performance. Assuming that a Bayesian reference model describes the true distribution of future data well, the theoretically preferred usage of the reference model is to project its predictive distribution to a reduced model, leading to projection predictive variable selection approach. We analyse how much the great performance of the projection predictive variable is due to the use of reference model and show that other variable selection methods can also be greatly improved by using the reference model as target instead of the original data. In several numerical experiments, we investigate the performance of the projective prediction approach as well as alternative variable selection methods with and without reference models. Our results indicate that the use of reference models generally translates into better and more stable variable selection.

Cracking propagation in expansive soils under desiccation and stabilization planning using Bayesian inference and Markov decision chains

Desiccation cracking endangers the stability of expansive soils subjected to cyclic moisture variations. In the current research, prominent cracking prediction models including linear, linear elastic, linear elastoplastic, and linear elastic fracture were studied. Then, Monte Carlo limit state functions were generated based on predictions. Results indicate that there is less than 5% chance of cracking for depths beyond 0.5, 6, 8, and 9 m as predicted by the linear elastoplastic, linear elastic, linear, and linear elastic fracture models, respectively. Moreover, a series of sensitivity analysis was performed to evaluate model and parameter uncertainties. Comparatively, it was found that the linear model exhibits the highest uncertainty while linear elastoplastic model possesses the least uncertainty thus yielding a reasonable prediction. Additionally, soil parameters including matric suction followed by dry density were identified to govern the overall cracking. Using Bayesian inference, numerous conditional probabilities of variation of soil properties were investigated. Then, several cracking probabilities under history of low to high matric suction and dry density were obtained. Accordingly, Monte Carlo Markov decision chains were established based on several ecofriendly and feasible stabilization policies and their performance was also evaluated. The obtained safety factors (SF) suggest that stabilization plans resulting in high moisture and dry density have the least likelihood of cracking with a SF equal to 5.1. However, stabilization policies having low dry density and moisture yield have the least SF of 0.39. Findings of this study can improve the decision-making processes for expansive soil stabilization by considering a variety of environmental conditional probabilities.

A Bayesian Approach to German Personal and Demonstrative Pronouns

When faced with an ambiguous pronoun, an addressee must interpret it by identifying a suitable referent. It has been proposed that the interpretation of pronouns can be captured using Bayes' Rule: P(referent|pronoun) ∝ P(pronoun|referent)P(referent). This approach has been successful in English and Mandarin Chinese. In this study, we further the cross-linguistic evidence for the Bayesian model by applying it to German personal and demonstrative pronouns, and provide novel quantitative support for the model by assessing model performance in a Bayesian statistical framework that allows implementation of a fully hierarchical structure, providing the most conservative estimates of uncertainty. Data from two story-continuation experiments showed that the Bayesian model overall made more accurate predictions for pronoun interpretation than production and next-mention biases separately. Furthermore, the model accounts for the demonstrative pronoun dieser as well as the personal pronoun, despite the demonstrative having different, and more rigid, resolution preferences.

No evidence for trait- and state-level urgency moderating the daily association between negative affect and subsequent alcohol use in two college samples

It remains unclear whether the negative reinforcement pathway to problematic drinking exists, and if so, for whom. One idea that has received some support recently is that people who tend to act impulsively in response to negative emotions (i.e. people high in negative urgency) may specifically respond to negative affect with increased alcohol consumption. We tested this idea in a preregistered secondary data analysis of two ecological momentary assessment studies using college samples. Participants (N = 226) reported on their current affective state multiple times per day and also the following morning reported alcohol use of the previous night. We assessed urgency both at baseline and during the momentary affect assessments. Results from our Bayesian model comparison procedure, which penalises increasing model complexity, indicate that no combination of the variables of interest (negative affect, urgency, and the respective interactions) outperformed a baseline model that included two known demographic predictors of alcohol use. A non-preregistered exploratory analysis provided some evidence for the effect of daily positive affect, positive urgency, as well as their interaction on subsequent alcohol use. Taken together, our results suggest that college students' drinking may be better described by a positive rather than negative reinforcement cycle.

Bayesian Estimation and Testing of a Beta Factor Model for Bounded Continuous Variables

The particularities of bounded data are often overlooked. This type of data is likely to display a pattern of skewness because of the existence of an upper and lower limit that cannot be exceeded. In the context of factor analysis, when variables are skewed in opposite directions, using normal-theory factor analysis might lead to over-factoring. We propose a Bayesian beta factor model to analyze doubly bounded data. A simulation study was conducted to evaluate the performance of the normal and beta factor models in the presence of skewed variables. Two Bayesian approaches to model evaluation methods are considered, posterior predictive checking and three information criterion measures (DIC, WAIC, and LOO). The number of estimated factors based on the Bayesian methods is compared for the normal and beta factor models. An application of the model using real data is also presented. We found that the beta factor model constitutes a suitable alternative to analyze data with a pattern of mixed skewness. Posterior predictive checking appears to be a viable option to select the optimal number of factors in Bayesian factor analysis.

Dynamic modelling of corporate credit ratings and defaults

In this article, we propose a longitudinal multivariate model for binary and ordinal outcomes to describe the dynamic relationship among firm defaults and credit ratings from various raters. The latent probability of default is modelled as a dynamic process which contains additive firm-specific effects, a latent systematic factor representing the business cycle and idiosyncratic observed and unobserved factors. The joint set-up also facilitates the estimation of a bias for each rater which captures changes in the rating standards of the rating agencies. Bayesian estimation techniques are employed to estimate the parameters of interest. Several models are compared based on their out-of-sample prediction ability and we find that the proposed model outperforms simpler specifications. The joint framework is illustrated on a sample of publicly traded US corporates which are rated by at least one of the credit rating agencies S&P, Moody's and Fitch during the period 1995–2014.

Spatial pattern analysis of line-segment data in ecology

The spatial analysis of linear features (lines and curves) is a challenging and rarely attempted problem in ecology. Existing methods are typically expressed in abstract mathematical formalism, making it difficult to assess their relevance and transferability into an ecological setting. We introduce a set of concrete and accessible methods to analyze the spatial patterning of line-segment data. The methods include Monte Carlo techniques based on a new generalization of Ripley's K -function and a class of line-segment processes that can be used to specify parametric models: parameters are estimated using maximum likelihood and models compared using information-theoretic principles. We apply the new methods to fallen tree (dead log) data collected from two 1-ha Australian tall eucalypt forest plots. Our results show that the spatial pattern of the fallen logs is best explained by plot-level spatial heterogeneity in combination with a slope-dependent nonuniform distribution of fallen-log orientations. These methods are of a general nature and are applicable to any line-segment data. In the context of forest ecology, the integration of fallen logs as linear structural features in a landscape with the point locations of living trees, and a quantification of their interactions, can yield new insights into the functional and structural role of tree fall in forest communities and their enduring post-mortem ecological legacy as spatially distributed decomposing logs.

Bayesian variable selection for understanding mixtures in environmental exposures

Social and environmental stressors are crucial factors in child development. However, there exists a multitude of measurable social and environmental factors-the effects of which may be cumulative, interactive, or null. Using a comprehensive cohort of children in North Carolina, we study the impact of social and environmental variables on 4th end-of-grade exam scores in reading and mathematics. To identify the essential factors that predict these educational outcomes, we design new tools for Bayesian linear variable selection using decision analysis. We extract a predictive optimal subset of explanatory variables by coupling a loss function with a novel model-based penalization scheme, which leads to coherent Bayesian decision analysis and empirically improves variable selection, estimation, and prediction on simulated data. The Bayesian linear model propagates uncertainty quantification to all predictive evaluations, which is important for interpretable and robust model comparisons. These predictive comparisons are conducted out-of-sample with a customized approximation algorithm that avoids computationally intensive model refitting. We apply our variable selection techniques to identify the joint collection of social and environmental stressors-and their interactions-that offer clear and quantifiable improvements in prediction of reading and mathematics exam scores.

Modeling Misretrieval and Feature Substitution in Agreement Attraction: A Computational Evaluation

We present computational modeling results based on a self-paced reading study investigating number attraction effects in Eastern Armenian. We implement three novel computational models of agreement attraction in a Bayesian framework and compare their predictive fit to the data using k-fold cross-validation. We find that our data are better accounted for by an encoding-based model of agreement attraction, compared to a retrieval-based model. A novel methodological contribution of our study is the use of comprehension questions with open-ended responses, so that both misinterpretation of the number feature of the subject phrase and misassignment of the thematic subject role of the verb can be investigated at the same time. We find evidence for both types of misinterpretation in our study, sometimes in the same trial. However, the specific error patterns in our data are not fully consistent with any previously proposed model.

Phylogenetic Typology

In this article we propose a novel method to estimate the frequency distribution of linguistic variables while controlling for statistical non-independence due to shared ancestry. Unlike previous approaches, our technique uses all available data, from language families large and small as well as from isolates, while controlling for different degrees of relatedness on a continuous scale estimated from the data. Our approach involves three steps: First, distributions of phylogenies are inferred from lexical data. Second, these phylogenies are used as part of a statistical model to estimate transition rates between parameter states. Finally, the long-term equilibrium of the resulting Markov process is computed. As a case study, we investigate a series of potential word-order correlations across the languages of the world.

Parsimonious model selection using information theory: a modified selection rule

Information-theoretic approaches to model selection, such as Akaike's information criterion (AIC) and cross validation, provide a rigorous framework to select among candidate hypotheses in ecology, yet the persistent concern of overfitting undermines the interpretation of inferred processes. A common misconception is that overfitting is due to the choice of criterion or model score, despite research demonstrating that selection uncertainty associated with score estimation is the predominant influence. Here we introduce a novel selection rule that identifies a parsimonious model by directly accounting for estimation uncertainty, while still retaining an information-theoretic interpretation. The new rule, which is a modification of the existing one-standard-error rule, mitigates overfitting and reduces the likelihood that spurious effects will be included in the selected model, thereby improving its inferential properties. We present the rule and illustrative examples in the context of maximum-likelihood estimation and Kullback-Leibler discrepancy, although the rule is applicable in a more general setting, including Bayesian model selection and other types of discrepancy.

Fast, Optimal, and Targeted Predictions using Parametrized Decision Analysis

Prediction is critical for decision-making under uncertainty and lends validity to statistical inference. With targeted prediction, the goal is to optimize predictions for specific decision tasks of interest, which we represent via functionals. Although classical decision analysis extracts predictions from a Bayesian model, these predictions are often difficult to interpret and slow to compute. Instead, we design a class of parametrized actions for Bayesian decision analysis that produce optimal, scalable, and simple targeted predictions. For a wide variety of action parametrizations and loss functions-including linear actions with sparsity constraints for targeted variable selection-we derive a convenient representation of the optimal targeted prediction that yields efficient and interpretable solutions. Customized out-of-sample predictive metrics are developed to evaluate and compare among targeted predictors. Through careful use of the posterior predictive distribution, we introduce a procedure that identifies a set of near-optimal, or acceptable targeted predictors, which provide unique insights into the features and level of complexity needed for accurate targeted prediction. Simulations demonstrate excellent prediction, estimation, and variable selection capabilities. Targeted predictions are constructed for physical activity data from the National Health and Nutrition Examination Survey (NHANES) to better predict and understand the characteristics of intraday physical activity.

On the Quantification of Model Uncertainty: A Bayesian Perspective

Issues of model selection have dominated the theoretical and applied statistical literature for decades. Model selection methods such as ridge regression, the lasso, and the elastic net have replaced ad hoc methods such as stepwise regression as a means of model selection. In the end, however, these methods lead to a single final model that is often taken to be the model considered ahead of time, thus ignoring the uncertainty inherent in the search for a final model. One method that has enjoyed a long history of theoretical developments and substantive applications, and that accounts directly for uncertainty in model selection, is Bayesian model averaging (BMA). BMA addresses the problem of model selection by not selecting a final model, but rather by averaging over a space of possible models that could have generated the data. The purpose of this paper is to provide a detailed and up-to-date review of BMA with a focus on its foundations in Bayesian decision theory and Bayesian predictive modeling. We consider the selection of parameter and model priors as well as methods for evaluating predictions based on BMA. We also consider important assumptions regarding BMA and extensions of model averaging methods to address these assumptions, particularly the method of Bayesian stacking. Simple empirical examples are provided and directions for future research relevant to psychometrics are discussed.

Efficient leave-one-out cross-validation for Bayesian non-factorized normal and Student-t models

Cross-validation can be used to measure a model’s predictive accuracy for the purpose of model comparison, averaging, or selection. Standard leave-one-out cross-validation (LOO-CV) requires that the observation model can be factorized into simple terms, but a lot of important models in temporal and spatial statistics do not have this property or are inefficient or unstable when forced into a factorized form. We derive how to efficiently compute and validate both exact and approximate LOO-CV for any Bayesian non-factorized model with a multivariate normal or Student-$$t$$ t distribution on the outcome values. We demonstrate the method using lagged simultaneously autoregressive (SAR) models as a case study.

A decision-theoretic approach for model interpretability in Bayesian framework

A salient approach to interpretable machine learning is to restrict modeling to simple models. In the Bayesian framework, this can be pursued by restricting the model structure and prior to favor interpretable models. Fundamentally, however, interpretability is about users’ preferences, not the data generation mechanism; it is more natural to formulate interpretability as a utility function. In this work, we propose an interpretability utility, which explicates the trade-off between explanation fidelity and interpretability in the Bayesian framework. The method consists of two steps. First, a reference model, possibly a black-box Bayesian predictive model which does not compromise accuracy, is fitted to the training data. Second, a proxy model from an interpretable model family that best mimics the predictive behaviour of the reference model is found by optimizing the interpretability utility function. The approach is model agnostic—neither the interpretable model nor the reference model are restricted to a certain class of models—and the optimization problem can be solved using standard tools. Through experiments on real-word data sets, using decision trees as interpretable models and Bayesian additive regression models as reference models, we show that for the same level of interpretability, our approach generates more accurate models than the alternative of restricting the prior. We also propose a systematic way to measure stability of interpretabile models constructed by different interpretability approaches and show that our proposed approach generates more stable models.

A Conceptual Introduction to Bayesian Model Averaging

Many statistical scenarios initially involve several candidate models that describe the data-generating process. Analysis often proceeds by first selecting the best model according to some criterion and then learning about the parameters of this selected model. Crucially, however, in this approach the parameter estimates are conditioned on the selected model, and any uncertainty about the model-selection process is ignored. An alternative is to learn the parameters for all candidate models and then combine the estimates according to the posterior probabilities of the associated models. This approach is known as Bayesian model averaging (BMA). BMA has several important advantages over all-or-none selection methods, but has been used only sparingly in the social sciences. In this conceptual introduction, we explain the principles of BMA, describe its advantages over all-or-none model selection, and showcase its utility in three examples: analysis of covariance, meta-analysis, and network analysis.

A Bayesian decision-theoretic approach to incorporate preclinical information into phase I oncology trials

Leveraging preclinical animal data for a phase I oncology trial is appealing yet challenging. In this paper, we use animal data to improve decision-making in a model-based dose-escalation procedure. We make a proposal for how to measure and address a prior-data conflict in a sequential study with a small sample size. Animal data are incorporated via a robust two-component mixture prior for the parameters of the human dose-toxicity relationship. The weights placed on each component of the prior are chosen empirically and updated dynamically as the trial progresses and more data accrue. After completion of each cohort, we use a Bayesian decision-theoretic approach to evaluate the predictive utility of the animal data for the observed human toxicity outcomes, reflecting the degree of agreement between dose-toxicity relationships in animals and humans. The proposed methodology is illustrated through several data examples and an extensive simulation study.

A Pan-Cancer and Polygenic Bayesian Hierarchical Model for the Effect of Somatic Mutations on Survival

We built a novel Bayesian hierarchical survival model based on the somatic mutation profile of patients across 50 genes and 27 cancer types. The pan-cancer quality allows for the model to “borrow” information across cancer types, motivated by the assumption that similar mutation profiles may have similar (but not necessarily identical) effects on survival across different tissues of origin or tumor types. The effect of a mutation at each gene was allowed to vary by cancer type, whereas the mean effect of each gene was shared across cancers. Within this framework, we considered 4 parametric survival models (normal, log-normal, exponential, and Weibull), and we compared their performance via a cross-validation approach in which we fit each model on training data and estimate the log-posterior predictive likelihood on test data. The log-normal model gave the best fit, and we investigated the partial effect of each gene on survival via a forward selection procedure. Through this we determined that mutations at TP53 and FAT4 were together the most useful for predicting patient survival. We validated the model via simulation to ensure that our algorithm for posterior computation gave nominal coverage rates. The code used for this analysis can be found at https://github.com/sarahsamorodnitsky/Pan-Cancer-Survival-Modeling.git, and the results are summarized at http://ericfrazerlock.com/surv_figs/SurvivalDisplay.html.

Simultaneous parameter estimation and variable selection via the logit-normal continuous analogue of the spike-and-slab prior

We introduce a Bayesian prior distribution, the logit-normal continuous analogue of the spike-and-slab, which enables flexible parameter estimation and variable/model selection in a variety of settings. We demonstrate its use and efficacy in three case studies—a simulation study and two studies on real biological data from the fields of metabolomics and genomics. The prior allows the use of classical statistical models, which are easily interpretable and well known to applied scientists, but performs comparably to common machine learning methods in terms of generalizability to previously unseen data.

On the marginal likelihood and cross-validation

In Bayesian statistics, the marginal likelihood, also known as the evidence, is used to evaluate model fit as it quantifies the joint probability of the data under the prior. In contrast, non-Bayesian models are typically compared using cross-validation on held-out data, either through $k$-fold partitioning or leave-$p$-out subsampling. We show that the marginal likelihood is formally equivalent to exhaustive leave-$p$-out crossvalidation averaged over all values of $p$ and all held-out test sets when using the log posterior predictive probability as the scoring rule. Moreover, the log posterior predictive score is the only coherent scoring rule under data exchangeability. This offers new insight into the marginal likelihood and cross-validation, and highlights the potential sensitivity of the marginal likelihood to the choice of the prior. We suggest an alternative approach using cumulative cross-validation following a preparatory training phase. Our work has connections to prequential analysis and intrinsic Bayes factors, but is motivated in a different way.

Bayesian model selection for spatial capture-recapture models

A vast amount of ecological knowledge generated over the past two decades has hinged upon the ability of model selection methods to discriminate among various ecological hypotheses. The last decade has seen the rise of Bayesian hierarchical models in ecology. Consequently, commonly used tools, such as the AIC, become largely inapplicable and there appears to be no consensus about a particular model selection tool that can be universally applied. We focus on a specific class of competing Bayesian spatial capture-recapture (SCR) models and apply and evaluate some of the recommended Bayesian model selection tools: (1) Bayes Factor-using (a) Gelfand-Dey and (b) harmonic mean methods, (2) Deviance Information Criterion (DIC), (3) Watanabe-Akaike's Information Criterion (WAIC) and (4) posterior predictive loss criterion. In all, we evaluate 25 variants of model selection tools in our study. We evaluate these model selection tools from the standpoint of selecting the "true" model and parameter estimation. In all, we generate 120 simulated data sets using the true model and assess the frequency with which the true model is selected and how well the tool estimates N (population size), a parameter of much importance to ecologists. We find that when information content is low in the data, no particular model selection tool can be recommended to help realize, simultaneously, both the goals of model selection and parameter estimation. But, in general (when we consider both the objectives together), we recommend the use of our application of the Bayes Factor (Gelfand-Dey with MAP approximation) for Bayesian SCR models. Our study highlights the point that although new model selection tools are emerging (e.g., WAIC) in the applied statistics literature, those tools based on sound theory even under approximation may still perform much better.

Correspondence between thermodynamics and inference

We expand upon a natural analogy between Bayesian statistics and statistical physics in which sample size corresponds to inverse temperature. This analogy motivates the definition of two statistical quantities: a learning capacity and a Gibbs entropy. The analysis of the learning capacity, corresponding to the heat capacity in thermal physics, leads to insight into the mechanism of learning and explains why some models have anomalously high learning performance. We explore the properties of the learning capacity in a number of examples, including a sloppy model. Next, we propose that the Gibbs entropy provides a natural device for counting distinguishable distributions in the context of Bayesian inference. We use this device to define a generalized principle of indifference in which every distinguishable model is assigned equal a priori probability. This principle results in a solution to a long-standing problem in Bayesian inference: the definition of an objective or uninformative prior. A key characteristic of this approach is that it can be applied to analyses where the model dimension is unknown and circumvents the automatic rejection of higher-dimensional models in Bayesian inference.

Introduction to Bayesian Inference for Psychology

We introduce the fundamental tenets of Bayesian inference, which derive from two basic laws of probability theory. We cover the interpretation of probabilities, discrete and continuous versions of Bayes' rule, parameter estimation, and model comparison. Using seven worked examples, we illustrate these principles and set up some of the technical background for the rest of this special issue of Psychonomic Bulletin & Review. Supplemental material is available via https://osf.io/wskex/ .