Do efficacy results obtained from randomized controlled trials translate to effectiveness data from observational studies for relapsing-remitting multiple sclerosis?

BACKGROUND

Randomized controlled trials are considered the gold standard in regulatory decision making, as observational studies are known to have important methodological limitations. However, real-world evidence may be helpful in specific situations. This review investigates how the effect estimates obtained from randomized controlled trials compare to those obtained from observational studies, using drug therapy for relapsing-remitting multiple sclerosis as an example.

STUDY DESIGN AND SETTING

A systematic review of randomized controlled trials and observational studies was conducted. The primary outcome was the annualized relapse rate. Using (network) meta-analysis together with posterior predictive distributions, the drug-specific rate ratios from the network of randomized controlled trials were compared with those from the network of observational studies.

RESULTS

Effect estimates from 26 observational studies showed greater magnitudes and were less precise compared to estimates obtained from 21 randomized controlled trials. Twenty of the 28 treatment comparisons between designs had similar rate ratios. Seven inconsistencies in observed rate ratios could be attributed to two specific disease-modifying therapies.

CONCLUSION

In this case study, estimates from observational studies predominantly agreed with estimates from randomized controlled trials given their posterior predictive distributions. Multiple observational studies together may therefore supplement additional pivotal randomized controlled trials in relapsing-remitting multiple sclerosis, for instance facilitating the extrapolation of trial results to the broader patient population.

Improved RNA stability estimation through Bayesian modeling reveals most Salmonella transcripts have subminute half-lives

RNA decay is a crucial mechanism for regulating gene expression in response to environmental stresses. In bacteria, RNA-binding proteins (RBPs) are known to be involved in posttranscriptional regulation, but their global impact on RNA half-lives has not been extensively studied. To shed light on the role of the major RBPs ProQ and CspC/E in maintaining RNA stability, we performed RNA sequencing of Salmonella enterica over a time course following treatment with the transcription initiation inhibitor rifampicin (RIF-seq) in the presence and absence of these RBPs. We developed a hierarchical Bayesian model that corrects for confounding factors in rifampicin RNA stability assays and enables us to identify differentially decaying transcripts transcriptome-wide. Our analysis revealed that the median RNA half-life in Salmonella in early stationary phase is less than 1 min, a third of previous estimates. We found that over half of the 500 most long-lived transcripts are bound by at least one major RBP, suggesting a general role for RBPs in shaping the transcriptome. Integrating differential stability estimates with cross-linking and immunoprecipitation followed by RNA sequencing (CLIP-seq) revealed that approximately 30% of transcripts with ProQ binding sites and more than 40% with CspC/E binding sites in coding or 3' untranslated regions decay differentially in the absence of the respective RBP. Analysis of differentially destabilized transcripts identified a role for ProQ in the oxidative stress response. Our findings provide insights into posttranscriptional regulation by ProQ and CspC/E, and the importance of RBPs in regulating gene expression.

The Expected Behaviors of Posterior Predictive Tests and Their Unexpected Interpretation

Poor fit between models of sequence or trait evolution and empirical data is known to cause biases and lead to spurious conclusions about evolutionary patterns and processes. Bayesian posterior prediction is a flexible and intuitive approach for detecting such cases of poor fit. However, the expected behavior of posterior predictive tests has never been characterized for evolutionary models, which is critical for their proper interpretation. Here, we show that the expected distribution of posterior predictive P-values is generally not uniform, in contrast to frequentist P-values used for hypothesis testing, and extreme posterior predictive P-values often provide more evidence of poor fit than typically appreciated. Posterior prediction assesses model adequacy under highly favorable circumstances, because the model is fitted to the data, which leads to expected distributions that are often concentrated around intermediate values. Nonuniform expected distributions of P-values do not pose a problem for the application of these tests, however, and posterior predictive P-values can be interpreted as the posterior probability that the fitted model would predict a dataset with a test statistic value as extreme as the value calculated from the observed data.

Seasonal habitat-use patterns of large mammals in a human-dominated landscape

Large mammals in temperate climates typically display seasonal patterns of habitat use. However, these patterns are often overlooked because large mammals are usually surveyed at annual intervals. In addition, most studies focus on a single species and ignore other species with which the focal species could interact. Knowing seasonal patterns of habitat use in multiple species and understanding factors that cause these patterns can provide further detail on population dynamics and guide effective conservation planning. Here, using dynamic occupancy modeling, we analyze 11 years of camera-trap data collected in northwestern Anatolia, Turkey, to investigate seasonal habitat use of 8 large-mammal species: Brown Bear (Ursus arctos), Eurasian Lynx (Lynx lynx), Gray Wolf (Canis lupus), Red Fox (Vulpes vulpes), Wild Boar (Sus scrofa), Roe Deer (Capreolus capreolus), European Hare (Lepus europaeus), and Red Deer (Cervus elaphus). For each species, we study the strength of seasonality in habitat use and its dependence on human population density and elevation, which have been shown to affect distributions of species in the region. Although all species exhibited seasonality in habitat use, the strength of this seasonality varied among species; it was strongest in Wild Boar, Roe Deer, and Brown Bear. Moreover, except for Brown Bear, all species tended to avoid sites close to humans. The species responded differently to changing elevation; increasing elevation had both positive and negative effects on species-specific colonization and desertion probabilities, and these effects were likely related to either feeding habits or tendency to avoid humans. These results indicate that seasonality should be taken into consideration in population studies. However, because species differ, seasonality patterns should be identified separately for each species of interest, as differences in these patterns can explain the underlying dynamics of habitat-use patterns more accurately.

Bayesian inference for a principal stratum estimand on recurrent events truncated by death

Recurrent events are often important endpoints in randomized clinical trials. For example, the number of recurrent disease-related hospitalizations may be considered as a clinically meaningful endpoint in cardiovascular studies. In some settings, the recurrent event process may be terminated by an event such as death, which makes it more challenging to define and estimate a causal treatment effect on recurrent event endpoints. In this paper, we focus on the principal stratum estimand, where the treatment effect of interest on recurrent events is defined among subjects who would be alive regardless of the assigned treatment. For the estimation of the principal stratum effect in randomized clinical trials, we propose a Bayesian approach based on a joint model of the recurrent event and death processes with a frailty term accounting for within-subject correlation. We also present Bayesian posterior predictive check procedures for assessing the model fit. The proposed approaches are demonstrated in the randomized Phase III chronic heart failure trial PARAGON-HF (NCT01920711).

Evaluating the Feasibility of Emotion Expressions in Avatars Created From Real Person Photos: Pilot Web-Based Survey of Virtual Reality Software

BACKGROUND

The availability and potential of virtual reality (VR) has led to an increase of its application. VR is suggested to be helpful in training elements of social competence but with an emphasis on interventions being tailored. Recognizing facial expressions is an important social skill and thus a target for training. Using VR in training these skills could have advantages over desktop alternatives. Children with autism, for instance, appear to prefer avatars over real images when assessing facial expressions. Available software provides the opportunity to transform profile pictures into avatars, thereby giving the possibility of tailoring according to an individual's own environment. However, the emotions provided by such software should be validated before application.

OBJECTIVE

Our aim was to investigate whether available software is a quick, easy, and viable way of providing emotion expressions in avatars transformed from real images.

METHODS

A total of 401 participants from a general population completed a survey on the web containing 27 different images of avatars transformed, using a software, from real images. We calculated the reliability of each image and their level of difficulty using a structural equation modeling approach. We used Bayesian confirmatory factor analysis testing under a multidimensional first-order correlated factor structure where faces showing the same emotions represented a latent variable.

RESULTS

Few emotions were correctly perceived and rated as higher than other emotions. The factor loadings indicating the discrimination of the image were around 0.7, which means 49% shared variance with the latent factor that the face is linked with. The standardized thresholds indicating the difficulty level of the images are mostly around average, and the highest correlation is between faces showing happiness and anger.

CONCLUSIONS

Only using a software to transform profile pictures to avatars is not sufficient to provide valid emotion expressions. Adjustments are needed to increase faces' discrimination (eg, increasing reliabilities). The faces showed average levels of difficulty, meaning that they are neither very difficult nor very easy to perceive, which fits a general population. Adjustments should be made for specific populations and when applying this technology in clinical practice.

Pallidal neuromodulation of the explore/exploit trade-off in decision-making

Every decision that we make involves a conflict between exploiting our current knowledge of an action's value or exploring alternative courses of action that might lead to a better, or worse outcome. The sub-cortical nuclei that make up the basal ganglia have been proposed as a neural circuit that may contribute to resolving this explore-exploit 'dilemma'. To test this hypothesis, we examined the effects of neuromodulating the basal ganglia's output nucleus, the globus pallidus interna, in patients who had undergone deep brain stimulation (DBS) for isolated dystonia. Neuromodulation enhanced the number of exploratory choices to the lower value option in a two-armed bandit probabilistic reversal-learning task. Enhanced exploration was explained by a reduction in the rate of evidence accumulation (drift rate) in a reinforcement learning drift diffusion model. We estimated the functional connectivity profile between the stimulating DBS electrode and the rest of the brain using a normative functional connectome derived from heathy controls. Variation in the extent of neuromodulation induced exploration between patients was associated with functional connectivity from the stimulation electrode site to a distributed brain functional network. We conclude that the basal ganglia's output nucleus, the globus pallidus interna, can adaptively modify decision choice when faced with the dilemma to explore or exploit.

Satisfaction with one's job and working at home in the COVID-19 pandemic: A two-wave study

As greater numbers of people have worked at home during the COVID-19 pandemic, workers, organisations and policy makers have begun considering the benefits of a sustained move towards homeworking, with workers' satisfaction with homeworking often cited as a key driver. But is satisfaction with homeworking that relevant to workers' overall job satisfaction? In this study, we examine whether job and homeworking satisfaction are predicted by different demands and resources, namely, those well established in the job design literature (workload, job autonomy and social support) for the former and those specific to the context of homeworking (loneliness, work-nonwork interference, work-nonwork interference and adequacy of homeworking environment) for the latter. We also explore whether homeworking satisfaction mediates the relationship between homeworking demands and resources and job satisfaction. Findings of a study of university workers during the COVID-19 pandemic (N = 753 in Phase 1, 471 in Phase 2) support our expectations about the domain-specific nature of the predictors of job and homeworking satisfaction, autonomy is positively related to job satisfaction, while loneliness, nonwork-to-work interference and inadequate homeworking environment are negatively related to homeworking satisfaction. Results also support the argument that satisfaction with homeworking mediates the relationship between homeworking factors and job satisfaction, reinforcing the value of differentiating the two concepts.

Modeling the Marked Presence-only Data: A Case Study of Estimating the Female Sex Worker Size in Malawi

Certain subpopulations like female sex workers (FSW), men who have sex with men (MSM), and people who inject drugs (PWID) often have higher prevalence of HIV/AIDS and are difficult to map directly due to stigma, discrimination, and criminalization. Fine-scale mapping of those populations contributes to the progress towards reducing the inequalities and ending the AIDS epidemic. In 2016 and 2017, the PLACE surveys were conducted at 3,290 venues in 20 out of the total 28 districts in Malawi to estimate the FSW sizes. These venues represent a presence-only data set where, instead of knowing both where people live and do not live (presence-absence data), only information about visited locations is available. In this study, we develop a Bayesian model for presence-only data and utilize the PLACE data to estimate the FSW size and uncertainty interval at a 1.5 × 1.5-km resolution for all of Malawi. The estimates can also be aggregated to any desirable level (city/district/region) for implementing targeted HIV prevention and treatment programs in FSW communities, which have been successful in lowering the incidence of HIV and other sexually transmitted infections.

Cognitive Training Improves Disconnected Limbs' Mental Representation and Peripersonal Space after Spinal Cord Injury

Paraplegia following spinal cord injury (SCI) affects the mental representation and peripersonal space of the paralysed body parts (i.e., lower limbs). Physical rehabilitation programs can improve these aspects, but the benefits are mostly partial and short-lasting. These limits could be due to the absence of trainings focused on SCI-induced cognitive deficits combined with traditional physical rehabilitation. To test this hypothesis, we assessed in 15 SCI-individuals the effects of adding cognitive recovery protocols (motor imagery–MI) to standard physical rehabilitation programs (Motor + MI training) on mental body representations and space representations, with respect to physical rehabilitation alone (control training). Each training comprised at least eight sessions administered over two weeks. The status of participants’ mental body representation and peripersonal space was assessed at three time points: before the training (T0), after the training (T1), and in a follow-up assessment one month later (T2). The Motor + MI training induced short-term recovery of peripersonal space that however did not persist at T2. Body representation showed a slower neuroplastic recovery at T2, without differences between Motor and the Motor + MI. These results show that body and space representations are plastic after lesions, and open new rehabilitation perspectives.

A multivariate statistical approach to predict COVID-19 count data with epidemiological interpretation and uncertainty quantification

For the analysis of COVID‐19 pandemic data, we propose Bayesian multinomial and Dirichlet‐multinomial autoregressive models for time‐series of counts of patients in mutually exclusive and exhaustive observational categories, defined according to the severity of the patient status and the required treatment. Categories include hospitalized in regular wards (H) and in intensive care units (ICU), together with deceased (D) and recovered (R). These models explicitly formulate assumptions on the transition probabilities between these categories across time, thanks to a flexible formulation based on parameters that a priori follow normal distributions, possibly truncated to incorporate specific hypotheses having an epidemiological interpretation. The posterior distribution of model parameters and the transition matrices are estimated by a Markov chain Monte Carlo algorithm that also provides predictions and allows us to compute the reproduction number Rt. All estimates and predictions are endowed with an accuracy measure obtained thanks to the Bayesian approach. We present results concerning data collected during the first wave of the pandemic in Italy and Lombardy and study the effect of nonpharmaceutical interventions. Suitable discrepancy measures defined to check and compare models show that the Dirichlet‐multinomial model has an adequate fit and provides good predictive performance in particular for H and ICU patients.

‘Unscrambling’ the drivers of egg production in Agassiz’s desert tortoise: climate and individual attributes predict reproductive output

The ‘bet hedging’ life history strategy of long-lived iteroparous species reduces short-term reproductive output to minimize the risk of reproductive failure over a lifetime. For desert-dwelling ectotherms living in variable and unpredictable environments, reproductive output is further influenced by precipitation and temperature via effects on food availability and limits on activity. We assembled multiple (n = 12) data sets on egg production for the threatened Agassiz’s desert tortoise Gopherus agassizii across its range and used these data to build a range-wide predictive model of annual reproductive output as a function of annual weather variation and individual-level attributes (body size and prior-year reproductive status). Climate variables were more robust predictors of reproductive output than individual-level attributes, with overall reproductive output positively related to prior-year precipitation and an earlier start to the spring activity season, and negatively related to spring temperature extremes (monthly temperature range in March-April). Reproductive output was highest for individuals with larger body sizes that reproduced in the previous year. Expected annual reproductive output from 1990-2018 varied from 2-5 to 6-12 eggs female-1 yr-1 , with a weak decline in expected reproductive output over this time (p = 0.02). Climate-driven environmental variation in expected reproductive output was highly correlated across all 5 Recovery Units for this species (Pearson’s r > 0.9). Overall, our model suggests that climate change could strongly impact the reproductive output of Agassiz’s desert tortoise, and could have a negative population-level effect if precipitation is significantly reduced across the species’ range as predicted under some climate models.

Modeling Within-Item Dependencies in Parallel Data on Test Responses and Brain Activation

In this paper, we propose a joint modeling approach to analyze dependency in parallel response data. We define two types of dependency: higher-level dependency and within-item conditional dependency. While higher-level dependency can be estimated with common latent variable modeling approaches, within-item conditional dependency is a unique kind of information that is often not captured with extant methods, despite its potential to shed new insights into the relationship between the two types of response data. We differentiate three ways of modeling within-item conditional dependency by conditioning on raw values, expected values, or residual values of the response data, which have different implications in terms of response processes. The proposed approach is illustrated with the example of analyzing parallel data on response accuracy and brain activations from a Theory of Mind assessment. The consequence of ignoring within-item conditional dependency is investigated with empirical and simulation studies in comparison to conventional dependency analysis that focuses exclusively on relationships between latent variables.

Rapid evolution at the Drosophila telomere: transposable element dynamics at an intrinsically unstable locus

Drosophila telomeres have been maintained by three families of active transposable elements (TEs), HeT-A, TAHRE, and TART, collectively referred to as HTTs, for tens of millions of years, which contrasts with an unusually high degree of HTT interspecific variation. While the impacts of conflict and domestication are often invoked to explain HTT variation, the telomeres are unstable structures such that neutral mutational processes and evolutionary tradeoffs may also drive HTT evolution. We leveraged population genomic data to analyze nearly 10,000 HTT insertions in 85  Drosophila melanogaster genomes and compared their variation to other more typical TE families. We observe that occasional large-scale copy number expansions of both HTTs and other TE families occur, highlighting that the HTTs are, like their feral cousins, typically repressed but primed to take over given the opportunity. However, large expansions of HTTs are not caused by the runaway activity of any particular HTT subfamilies or even associated with telomere-specific TE activity, as might be expected if HTTs are in strong genetic conflict with their hosts. Rather than conflict, we instead suggest that distinctive aspects of HTT copy number variation and sequence diversity largely reflect telomere instability, with HTT insertions being lost at much higher rates than other TEs elsewhere in the genome. We extend previous observations that telomere deletions occur at a high rate, and surprisingly discover that more than one-third do not appear to have been healed with an HTT insertion. We also report that some HTT families may be preferentially activated by the erosion of whole telomeres, implying the existence of HTT-specific host control mechanisms. We further suggest that the persistent telomere localization of HTTs may reflect a highly successful evolutionary strategy that trades away a stable insertion site in order to have reduced impact on the host genome. We propose that HTT evolution is driven by multiple processes, with niche specialization and telomere instability being previously underappreciated and likely predominant.

Perceived Stress Can Mediate the Associations between a Lifestyle Intervention and Fat and Fast Food Intakes

This secondary analysis study addressed a gap of knowledge: whether perceived stress reduction created by a lifestyle intervention might serve as a mediator for reducing fat and fast food intakes in low-income overweight or obese mothers of young children. This analysis included 338 low-income overweight or obese mothers of young children who completed a phone interview immediately after the 16-week lifestyle intervention. Valid surveys were used to assess perceived stress and fat and fast food intakes. Composite indicator structural equation modeling was performed to test the mediation effects. The overall effect of the intervention was not significant for fat intake but was significant for fast food intake (B = −0.53, p < 0.05). When assessing the potential role of perceived stress as a mediator, the indirect effects of the intervention on fat (B = −0.39, p < 0.01) and fast food (B = −0.27, p < 0.01) intakes were both significant. Future dietary intervention studies aimed to reduce fat and fast food intakes in low-income overweight or obese mothers of young children might consider including practical strategies aimed at reducing perceived stress.

Prospective Associations of Child Maltreatment Subtypes With Adult Educational Attainment: Tests of Mediating Mechanisms Through School-Related Outcomes

This study tests a developmental cascades model in which childhood maltreatment is hypothesized to influence adult educational attainment by increasing attention problems and reducing successful school experiences during adolescence. Two path models tested direct and indirect associations of childhood maltreatment with adult educational attainment. Model 1 used three parent-reported subtypes of childhood maltreatment (physical/emotional abuse, sexual abuse, neglect). Model 2 added an agency-reported measure of childhood maltreatment. Both models detected indirect effects of childhood maltreatment subtypes on adult educational attainment through attention and school discipline problems, poor school engagement, and low extracurricular involvement. Model 2 also detected a direct effect of agency-reported childhood maltreatment on the adult outcome. Regardless of the type of maltreatment or data source used, school factors mediated the associations of childhood maltreatment on adult educational attainment. Promoting school engagement and reducing disciplinary referrals for maltreated youth could improve their educational attainment over the long term.

Latent likelihood ratio tests for assessing spatial kernels in epidemic models

One of the most important issues in the critical assessment of spatio-temporal stochastic models for epidemics is the selection of the transmission kernel used to represent the relationship between infectious challenge and spatial separation of infected and susceptible hosts. As the design of control strategies is often based on an assessment of the distance over which transmission can realistically occur and estimation of this distance is very sensitive to the choice of kernel function, it is important that models used to inform control strategies can be scrutinised in the light of observation in order to elicit possible evidence against the selected kernel function. While a range of approaches to model criticism is in existence, the field remains one in which the need for further research is recognised. In this paper, building on earlier contributions by the authors, we introduce a new approach to assessing the validity of spatial kernels—the latent likelihood ratio tests—which use likelihood-based discrepancy variables that can be used to compare the fit of competing models, and compare the capacity of this approach to detect model mis-specification with that of tests based on the use of infection-link residuals. We demonstrate that the new approach can be used to formulate tests with greater power than infection-link residuals to detect kernel mis-specification particularly when the degree of mis-specification is modest. This new tests avoid the use of a fully Bayesian approach which may introduce undesirable complications related to computational complexity and prior sensitivity.

Combining Multiband Remote Sensing and Hierarchical Distance Sampling to Establish Drivers of Bird Abundance

Information on habitat preferences is critical for the successful conservation of endangered species. For many species, especially those living in remote areas, we currently lack this information. Time and financial resources to analyze habitat use are limited. We aimed to develop a method to describe habitat preferences based on a combination of bird surveys with remotely sensed fine-scale land cover maps. We created a blended multiband remote sensing product from SPOT 6 and Landsat 8 data with a high spatial resolution. We surveyed populations of three bird species (Yellow-breasted Bunting Emberiza aureola, Ochre-rumped Bunting Emberiza yessoensis, and Black-faced Bunting Emberiza spodocephala) at a study site in the Russian Far East using hierarchical distance sampling, a survey method that allows to correct for varying detection probability. Combining the bird survey data and land cover variables from the remote sensing product allowed us to model population density as a function of environmental variables. We found that even small-scale land cover characteristics were predictable using remote sensing data with sufficient accuracy. The overall classification accuracy with pansharpened SPOT 6 data alone amounted to 71.3%. Higher accuracies were reached via the additional integration of SWIR bands (overall accuracy = 73.21%), especially for complex small-scale land cover types such as shrubby areas. This helped to reach a high accuracy in the habitat models. Abundances of the three studied bird species were closely linked to the proportion of wetland, willow shrubs, and habitat heterogeneity. Habitat requirements and population sizes of species of interest are valuable information for stakeholders and decision-makers to maximize the potential success of habitat management measures.

Experimental harvest regulations reveal that water availability during spring, not harvest, affects change in a waterfowl population

Population change is regulated by vital rates that are influenced by environmental conditions, demographic stochasticity, and, increasingly, anthropogenic effects. Habitat destruction and climate change threaten the future of many wildlife populations, and there are additional concerns regarding the effects of harvest rates on demographic components of harvested organisms. Further, many population managers strictly manage harvest of wild organisms to mediate population trends of these populations. The goal of our study was to decouple harvest and environmental variability in a closely monitored population of wild ducks in North America, where we experimentally regulated harvest independently of environmental variation over a period of 4 years. We used 9 years of capture-mark-recapture data to estimate breeding population size during the spring for a population of wood ducks in Nevada. We then assessed the effect of one environmental variable and harvest pressure on annual changes in the breeding population size. Climatic conditions influencing water availability were strongly positively related to population growth rates of wood ducks in our study system. In contrast, harvest regulations and harvest rates did not affect population growth rates. We suggest efforts to conserve waterfowl should focus on the effects of habitat loss in breeding areas and climate change, which will likely affect precipitation regimes in the future. We demonstrate the utility of capture-mark-recapture methods to estimate abundance of species which are difficult to survey and test the impacts of anthropogenic harvest and climate on populations. Finally, our results continue to add to the importance of experimentation in applied conservation biology, where we believe that continued experiments on nonthreatened species will be critically important as researchers attempt to understand how to quantify and mitigate direct anthropogenic impacts in a changing world.

Dynamic models of choice

Parameter estimation in evidence-accumulation models of choice response times is demanding of both the data and the user. We outline how to fit evidence-accumulation models using the flexible, open-source, R-based Dynamic Models of Choice (DMC) software. DMC provides a hands-on introduction to the Bayesian implementation of two popular evidence-accumulation models: the diffusion decision model (DDM) and the linear ballistic accumulator (LBA). It enables individual and hierarchical estimation, as well as assessment of the quality of a model's parameter estimates and descriptive accuracy. First, we introduce the basic concepts of Bayesian parameter estimation, guiding the reader through a simple DDM analysis. We then illustrate the challenges of fitting evidence-accumulation models using a set of LBA analyses. We emphasize best practices in modeling and discuss the importance of parameter- and model-recovery simulations, exploring the strengths and weaknesses of models in different experimental designs and parameter regions. We also demonstrate how DMC can be used to model complex cognitive processes, using as an example a race model of the stop-signal paradigm, which is used to measure inhibitory ability. We illustrate the flexibility of DMC by extending this model to account for mixtures of cognitive processes resulting from attention failures. We then guide the reader through the practical details of a Bayesian hierarchical analysis, from specifying priors to obtaining posterior distributions that encapsulate what has been learned from the data. Finally, we illustrate how the Bayesian approach leads to a quantitatively cumulative science, showing how to use posterior distributions to specify priors that can be used to inform the analysis of future experiments.

Dynamically borrowing strength from another study through shrinkage estimation

Meta-analytic methods may be used to combine evidence from different sources of information. Quite commonly, the normal–normal hierarchical model (NNHM) including a random-effect to account for between-study heterogeneity is utilized for such analyses. The same modeling framework may also be used to not only derive a combined estimate, but also to borrow strength for a particular study from another by deriving a shrinkage estimate. For instance, a small-scale randomized controlled trial could be supported by a non-randomized study, e.g. a clinical registry. This would be particularly attractive in the context of rare diseases. We demonstrate that a meta-analysis still makes sense in this extreme case, effectively based on a synthesis of only two studies, as illustrated using a recent trial and a clinical registry in Creutzfeld-Jakob disease. Derivation of a shrinkage estimate within a Bayesian random-effects meta-analysis may substantially improve a given estimate even based on only a single additional estimate while accounting for potential effect heterogeneity between the studies. Alternatively, inference may equivalently be motivated via a model specification that does not require a common overall mean parameter but considers the treatment effect in one study, and the difference in effects between the studies. The proposed approach is quite generally applicable to combine different types of evidence originating, e.g. from meta-analyses or individual studies. An application of this more general setup is provided in immunosuppression following liver transplantation in children.

A Simple Method for Comparing Complex Models: Bayesian Model Comparison for Hierarchical Multinomial Processing Tree Models Using Warp-III Bridge Sampling

Multinomial processing trees (MPTs) are a popular class of cognitive models for categorical data. Typically, researchers compare several MPTs, each equipped with many parameters, especially when the models are implemented in a hierarchical framework. A Bayesian solution is to compute posterior model probabilities and Bayes factors. Both quantities, however, rely on the marginal likelihood, a high-dimensional integral that cannot be evaluated analytically. In this case study, we show how Warp-III bridge sampling can be used to compute the marginal likelihood for hierarchical MPTs. We illustrate the procedure with two published data sets and demonstrate how Warp-III facilitates Bayesian model averaging.

Bayesian statistical approaches to evaluating cognitive models

Cognitive models aim to explain complex human behavior in terms of hypothesized mechanisms of the mind. These mechanisms can be formalized in terms of mathematical structures containing parameters that are theoretically meaningful. For example, in the case of perceptual decision making, model parameters might correspond to theoretical constructs like response bias, evidence quality, response caution, and the like. Formal cognitive models go beyond verbal models in that cognitive mechanisms are instantiated in terms of mathematics and they go beyond statistical models in that cognitive model parameters are psychologically interpretable. We explore three key elements used to formally evaluate cognitive models: parameter estimation, model prediction, and model selection. We compare and contrast traditional approaches with Bayesian statistical approaches to performing each of these three elements. Traditional approaches rely on an array of seemingly ad hoc techniques, whereas Bayesian statistical approaches rely on a single, principled, internally consistent system. We illustrate the Bayesian statistical approach to evaluating cognitive models using a running example of the Linear Ballistic Accumulator model of decision making (Brown SD, Heathcote A. The simplest complete model of choice response time: linear ballistic accumulation. Cogn Psychol 2008, 57:153-178). WIREs Cogn Sci 2018, 9:e1458. doi: 10.1002/wcs.1458 This article is categorized under: Neuroscience > Computation Psychology > Reasoning and Decision Making Psychology > Theory and Methods.

Thou Shalt Not Bear False Witness Against Null Hypothesis Significance Testing

Null hypothesis significance testing (NHST) has been the subject of debate for decades and alternative approaches to data analysis have been proposed. This article addresses this debate from the perspective of scientific inquiry and inference. Inference is an inverse problem and application of statistical methods cannot reveal whether effects exist or whether they are empirically meaningful. Hence, raising conclusions from the outcomes of statistical analyses is subject to limitations. NHST has been criticized for its misuse and the misconstruction of its outcomes, also stressing its inability to meet expectations that it was never designed to fulfil. Ironically, alternatives to NHST are identical in these respects, something that has been overlooked in their presentation. Three of those alternatives are discussed here (estimation via confidence intervals and effect sizes, quantification of evidence via Bayes factors, and mere reporting of descriptive statistics). None of them offers a solution to the problems that NHST is purported to have, all of them are susceptible to misuse and misinterpretation, and some bring around their own problems (e.g., Bayes factors have a one-to-one correspondence with p values, but they are entirely deprived of an inferential framework). Those alternatives also fail to cover a broad area of inference not involving distributional parameters, where NHST procedures remain the only (and suitable) option. Like knives or axes, NHST is not inherently evil; only misuse and misinterpretation of its outcomes needs to be eradicated.

Estimating cross-validatory predictive p-values with integrated importance sampling for disease mapping models

An important statistical task in disease mapping problems is to identify divergent regions with unusually high or low risk of disease. Leave-one-out cross-validatory (LOOCV) model assessment is the gold standard for estimating predictive p-values that can flag such divergent regions. However, actual LOOCV is time-consuming because one needs to rerun a Markov chain Monte Carlo analysis for each posterior distribution in which an observation is held out as a test case. This paper introduces a new method, called integrated importance sampling (iIS), for estimating LOOCV predictive p-values with only Markov chain samples drawn from the posterior based on a full data set. The key step in iIS is that we integrate away the latent variables associated the test observation with respect to their conditional distribution without reference to the actual observation. By following the general theory for importance sampling, the formula used by iIS can be proved to be equivalent to the LOOCV predictive p-value. We compare iIS and other three existing methods in the literature with two disease mapping datasets. Our empirical results show that the predictive p-values estimated with iIS are almost identical to the predictive p-values estimated with actual LOOCV and outperform those given by the existing three methods, namely, the posterior predictive checking, the ordinary importance sampling, and the ghosting method by Marshall and Spiegelhalter (2003). Copyright © 2017 John Wiley & Sons, Ltd.

Bayesian analysis of longitudinal multitrait-multimethod data with ordinal response variables

A new multilevel latent state graded response model for longitudinal multitrait-multimethod (MTMM) measurement designs combining structurally different and interchangeable methods is proposed. The model allows researchers to examine construct validity over time and to study the change and stability of constructs and method effects based on ordinal response variables. We show how Bayesian estimation techniques can address a number of important issues that typically arise in longitudinal multilevel MTMM studies and facilitates the estimation of the model presented. Estimation accuracy and the impact of between- and within-level sample sizes as well as different prior specifications on parameter recovery were investigated in a Monte Carlo simulation study. Findings indicate that the parameters of the model presented can be accurately estimated with Bayesian estimation methods in the case of low convergent validity with as few as 250 clusters and more than two observations within each cluster. The model was applied to well-being data from a longitudinal MTMM study, assessing the change and stability of life satisfaction and subjective happiness in young adults after high-school graduation. Guidelines for empirical applications are provided and advantages and limitations of a Bayesian approach to estimating longitudinal multilevel MTMM models are discussed.

A Bayesian spatial random effects model characterisation of tumour heterogeneity implemented using Markov chain Monte Carlo (MCMC) simulation

The focus of this study is the development of a statistical modelling procedure for characterising intra-tumour heterogeneity, motivated by recent clinical literature indicating that a variety of tumours exhibit a considerable degree of genetic spatial variability. A formal spatial statistical model has been developed and used to characterise the structural heterogeneity of a number of supratentorial primitive neuroectodermal tumours (PNETs), based on diffusion-weighted magnetic resonance imaging. Particular attention is paid to the spatial dependence of diffusion close to the tumour boundary, in order to determine whether the data provide statistical evidence to support the proposition that water diffusivity in the boundary region of some tumours exhibits a deterministic dependence on distance from the boundary, in excess of an underlying random 2D spatial heterogeneity in diffusion. Tumour spatial heterogeneity measures were derived from the diffusion parameter estimates obtained using a Bayesian spatial random effects model. The analyses were implemented using Markov chain Monte Carlo (MCMC) simulation. Posterior predictive simulation was used to assess the adequacy of the statistical model. The main observations are that the previously reported relationship between diffusion and boundary proximity remains observable and achieves statistical significance after adjusting for an underlying random 2D spatial heterogeneity in the diffusion model parameters. A comparison of the magnitude of the boundary-distance effect with the underlying random 2D boundary heterogeneity suggests that both are important sources of variation in the vicinity of the boundary. No consistent pattern emerges from a comparison of the boundary and core spatial heterogeneity, with no indication of a consistently greater level of heterogeneity in one region compared with the other. The results raise the possibility that DWI might provide a surrogate marker of intra-tumour genetic regional heterogeneity, which would provide a powerful tool with applications in both patient management and in cancer research.

The comparability of the universalism value over time and across countries in the European Social Survey: exact vs. approximate measurement invariance

Over the last decades, large international datasets such as the European Social Survey (ESS), the European Value Study (EVS) and the World Value Survey (WVS) have been collected to compare value means over multiple time points and across many countries. Yet analyzing comparative survey data requires the fulfillment of specific assumptions, i.e., that these values are comparable over time and across countries. Given the large number of groups that can be compared in repeated cross-national datasets, establishing measurement invariance has been, however, considered unrealistic. Indeed, studies which did assess it often failed to establish higher levels of invariance such as scalar invariance. In this paper we first introduce the newly developed approximate approach based on Bayesian structural equation modeling (BSEM) to assess cross-group invariance over countries and time points and contrast the findings with the results from the traditional exact measurement invariance test. BSEM examines whether measurement parameters are approximately (rather than exactly) invariant. We apply BSEM to a subset of items measuring the universalism value from the Portrait Values Questionnaire (PVQ) in the ESS. The invariance of this value is tested simultaneously across 15 ESS countries over six ESS rounds with 173,071 respondents and 90 groups in total. Whereas, the use of the traditional approach only legitimates the comparison of latent means of 37 groups, the Bayesian procedure allows the latent mean comparison of 73 groups. Thus, our empirical application demonstrates for the first time the BSEM test procedure on a particularly large set of groups.

Assessing Model Fit in Latent Class Analysis When Asymptotics Do Not Hold

The application of latent class (LC) analysis involves evaluating the LC model using goodness-of-fit statistics. To assess the misfit of a specified model, say with the Pearson chi-squared statistic, a p-value can be obtained using an asymptotic reference distribution. However, asymptotic p-values are not valid when the sample size is not large and/or the analyzed contingency table is sparse. Another problem is that for various other conceivable global and local fit measures, asymptotic distributions are not readily available. An alternative way to obtain the p-value for the statistic of interest is by constructing its empirical reference distribution using resampling techniques such as the parametric bootstrap or the posterior predictive check (PPC). In the current paper, we show how to apply the parametric bootstrap and two versions of the PPC to obtain empirical p-values for a number of commonly used global and local fit statistics within the context of LC analysis. The main difference between the PPC using test statistics and the parametric bootstrap is that the former takes into account parameter uncertainty. The PPC using discrepancies has the advantage that it is computationally much less intensive than the other two resampling methods. In a Monte Carlo study we evaluated Type I error rates and power of these resampling methods when used for global and local goodness-of-fit testing in LC analysis. Results show that both the bootstrap and the PPC using test statistics are generally good alternatives to asymptotic p-values and can also be used when (asymptotic) distributions are not known. Nominal Type I error rates were not met when sample size was small and the contingency table has many cells. Overall the PPC using test statistics was somewhat more conservative than the parametric bootstrap. We have also replicated previous research suggesting that the Pearson χ2 statistic should in many cases be preferred over the likelihood-ratio G2 statistic. Power to reject a model for which the number of LCs was one less than in the population was very high, unless sample size was small. When the contingency tables are very sparse, the total bivariate residual (TBVR) statistic, which is based on bivariate relationships, still had very high power, signifying its usefulness in assessing model fit.

Why preferring parametric forecasting to nonparametric methods?

A recent series of papers by Charles T. Perretti and collaborators have shown that nonparametric forecasting methods can outperform parametric methods in noisy nonlinear systems. Such a situation can arise because of two main reasons: the instability of parametric inference procedures in chaotic systems which can lead to biased parameter estimates, and the discrepancy between the real system dynamics and the modeled one, a problem that Perretti and collaborators call "the true model myth". Should ecologists go on using the demanding parametric machinery when trying to forecast the dynamics of complex ecosystems? Or should they rely on the elegant nonparametric approach that appears so promising? It will be here argued that ecological forecasting based on parametric models presents two key comparative advantages over nonparametric approaches. First, the likelihood of parametric forecasting failure can be diagnosed thanks to simple Bayesian model checking procedures. Second, when parametric forecasting is diagnosed to be reliable, forecasting uncertainty can be estimated on virtual data generated with the fitted to data parametric model. In contrast, nonparametric techniques provide forecasts with unknown reliability. This argumentation is illustrated with the simple theta-logistic model that was previously used by Perretti and collaborators to make their point. It should convince ecologists to stick to standard parametric approaches, until methods have been developed to assess the reliability of nonparametric forecasting.

Posterior predictive checking of multiple imputation models

Multiple imputation is gaining popularity as a strategy for handling missing data, but there is a scarcity of tools for checking imputation models, a critical step in model fitting. Posterior predictive checking (PPC) has been recommended as an imputation diagnostic. PPC involves simulating "replicated" data from the posterior predictive distribution of the model under scrutiny. Model fit is assessed by examining whether the analysis from the observed data appears typical of results obtained from the replicates produced by the model. A proposed diagnostic measure is the posterior predictive "p-value", an extreme value of which (i.e., a value close to 0 or 1) suggests a misfit between the model and the data. The aim of this study was to evaluate the performance of the posterior predictive p-value as an imputation diagnostic. Using simulation methods, we deliberately misspecified imputation models to determine whether posterior predictive p-values were effective in identifying these problems. When estimating the regression parameter of interest, we found that more extreme p-values were associated with poorer imputation model performance, although the results highlighted that traditional thresholds for classical p-values do not apply in this context. A shortcoming of the PPC method was its reduced ability to detect misspecified models with increasing amounts of missing data. Despite the limitations of posterior predictive p-values, they appear to have a valuable place in the imputer's toolkit. In addition to automated checking using p-values, we recommend imputers perform graphical checks and examine other summaries of the test quantity distribution.