Bayesian Hodges-Lehmann tests for statistical equivalence in the two-sample setting: Power analysis, type I error rates and equivalence boundary selection in biomedical research

Statistical Testing for Protein Equivalence Identifies Core Functional Modules Conserved across 360 Cancer Cell Lines and Presents a General Approach to Investigating Biological Systems

Quantitative proteomics has enhanced our capability to study protein dynamics and their involvement in disease using various techniques, including statistical testing, to discern the significant differences between conditions. While most focus is on what is different between conditions, exploring similarities can provide valuable insights. However, exploring similarities directly from the analyte level, such as proteins, genes, or metabolites, is not a standard practice and is not widely adopted. In this study, we propose a statistical framework called QuEStVar (Quantitative Exploration of Stability and Variability through statistical hypothesis testing), enabling the exploration of quantitative stability and variability of features with a combined statistical framework. QuEStVar utilizes differential and equivalence testing to expand statistical classifications of analytes when comparing conditions. We applied our method to an extensive data set of cancer cell lines and revealed a quantitatively stable core proteome across diverse tissues and cancer subtypes. The functional analysis of this set of proteins highlighted the molecular mechanism of cancer cells to maintain constant conditions of the tumorigenic environment via biological processes, including transcription, translation, and nucleocytoplasmic transport.

Bayesian Mendelian randomization with an interval causal null hypothesis: ternary decision rules and loss function calibration

We enhance the Bayesian Mendelian Randomization (MR) framework of Berzuini et al. (Biostatistics 21(1):86-101, 2018) by allowing for interval null causal hypotheses, where values of the causal effect parameter that fall within a user-specified interval of "practical equivalence" (ROPE) (Kruschke, Adv Methods Pract Psychol Sci 1(2):270-80, 2018) are regarded as equivalent to "no effect". We motivate this move in the context of MR analysis. In this approach, the decision over the hypothesis test is taken on the basis of the Bayesian posterior odds for the causal effect parameter falling within the ROPE. We allow the causal effect parameter to have a mixture prior, with components corresponding to the null and the alternative hypothesis. Inference is performed via Markov chain Monte Carlo (MCMC) methods. We speed up the calculations by fitting to the data a simpler model than the intended, "true", one. We recover a set of samples from the "true" posterior distribution by weighted importance resampling of the MCMC-generated samples. From the final samples we obtain a simulation consistent estimate of the desired posterior odds, and ultimately of the Bayes factor for the interval-valued null hypothesis, [Formula: see text], vs [Formula: see text]. In those situations where the posterior odds is neither large nor small enough, we allow for an uncertain outcome of the test decision, thereby moving to a ternary decision logic. Finally, we present an approach to calibration of the proposed method via loss function. We illustrate the method with the aid of a study of the causal effect of obesity on risk of juvenile myocardial infarction based on a unique prospective dataset.

A Bayesian approach for evaluating equivalence over multiple groups, and comparison with frequentist tost

Manufacturing and testing of pharmaceutical products frequently occur in multiple facilities within a company's network. It is of interest to demonstrate equivalence among the alternative testing/manufacturing facilities to ensure product consistency and quality regardless of the facility where it was manufactured/tested. In the Frequentist framework, equivalence testing is well established when comparing two labs or manufacturing facilities; however, when considering more than two labs or production sites, the Frequentist approach may not always offer appropriate or interpretable estimates for demonstrating equivalence among all of them simultaneously. This paper demonstrates the utility of Bayesian methods to the equivalence assessment of multiple groups means, with a comparison against traditional Frequentist methods. We conclude that a Bayesian strategy is very useful for addressing the problem of multi-group equivalence. While it is not our intention to argue that Bayesian methods should always replace Frequentist ones, we show that among the advantages of a Bayesian analysis is that it provides a more nuanced understanding of the degree of similarity among sites than the hypothesis testing underpinning the Frequentist approach.

Learning emotional dialects: A British population study of cross-cultural communication

The aim of the current research was to explore whether we can improve the recognition of cross-cultural freely-expressed emotional faces in British participants. We tested several methods for improving the recognition of freely-expressed emotional faces, such as different methods for presenting other-culture expressions of emotion from individuals from Chile, New Zealand and Singapore in two experimental stages. In the first experimental stage, in phase one, participants were asked to identify the emotion of cross-cultural freely-expressed faces. In the second phase, different cohorts were presented with interactive side-by-side, back-to-back and dynamic morphing of cross-cultural freely-expressed emotional faces, and control conditions. In the final phase, we repeated phase one using novel stimuli. We found that all non-control conditions led to recognition improvements. Morphing was the most effective condition for improving the recognition of cross-cultural emotional faces. In the second experimental stage, we presented morphing to different cohorts including own-to-other and other-to-own freely-expressed cross-cultural emotional faces and neutral-to-emotional and emotional-to-neutral other-culture freely-expressed emotional faces. All conditions led to recognition improvements and the presentation of freely-expressed own-to-other cultural-emotional faces provided the most effective learning. These findings suggest that training can improve the recognition of cross-cultural freely-expressed emotional expressions.

Reducing the false discovery rate of preclinical animal research with Bayesian statistical decision criteria

The success of preclinical research hinges on exploratory and confirmatory animal studies. Traditional null hypothesis significance testing is a common approach to eliminate the chaff from a collection of drugs, so that only the most promising treatments are funneled through to clinical research phases. Balancing the number of false discoveries and false omissions is an important aspect to consider during this process. In this paper, we compare several preclinical research pipelines, either based on null hypothesis significance testing or based on Bayesian statistical decision criteria. We build on a recently published large-scale meta-analysis of reported effect sizes in preclinical animal research and elicit a non-informative prior distribution under which both approaches are compared. After correcting for publication bias and shrinkage of effect sizes in replication studies, simulations show that (i) a shift towards statistical approaches which explicitly incorporate the minimum clinically important difference reduces the false discovery rate of frequentist approaches and (ii) a shift towards Bayesian statistical decision criteria can improve the reliability of preclinical animal research by reducing the number of false-positive findings. It is shown that these benefits hold while keeping the number of experimental units low which are required for a confirmatory follow-up study. Results show that Bayesian statistical decision criteria can help in improving the reliability of preclinical animal research and should be considered more frequently in practice.

A Case Study for Critical Reagent Qualification for Ligand Binding Assays Using Equivalence Test Methodology

Qualifying critical reagents in ligand binding assays by parallel testing of current and candidate reagent lots is recommended by regulatory agencies and industry groups, but specific guidance on the format of reagent qualification experiments is limited. Equivalence testing is a statistically sound approach that is consistent with the objective of critical reagent qualification. We present power analysis for equivalence regions ranging from 1.25- to 1.5-fold multiples of the GM ratio (centered on 1) of current and candidate lots, over a range of assay variability from 5 to 30% coefficient of variation (CV). A 1.25-fold equivalence region can be tested using 6 to 12 plates per lot for assays with up to 15% CV but is not practical for more variable assays. For these assays, wider equivalence regions are justified so long as care is taken to avoid assay drift and the assay remains suitable for the intended use. The equivalence test method is illustrated using historical data from passing and failing reagent qualification experiments. Simulation analysis was performed to support the design of qualification experiments using 6, 12, or 18 plates per lot over a broad range of assay variability. A challenge in implementing the equivalence test approach is selecting an appropriate equivalence region. Equivalence regions providing 90% power using 12 plates/lot were consistent with 1.5σ bounds, which are recommended for equivalence testing of critical quality attributes of biosimilars.

Understanding Randomness on a Molecular Level: A Diagnostic Tool

Undergraduate biology students' molecular-level understanding of stochastic (also referred to as random or noisy) processes found in biological systems is often limited to those examples discussed in class. Therefore, students frequently display little ability to accurately transfer their knowledge to other contexts. Furthermore, elaborate tools to assess students' understanding of these stochastic processes are missing, despite the fundamental nature of this concept and the increasing evidence demonstrating its importance in biology. Thus, we developed the Molecular Randomness Concept Inventory (MRCI), an instrument composed of nine multiple-choice questions based on students' most prevalent misconceptions, to quantify students' understanding of stochastic processes in biological systems. The MRCI was administered to 67 first-year natural science students in Switzerland. The psychometric properties of the inventory were analyzed using classical test theory and Rasch modeling. Moreover, think-aloud interviews were conducted to ensure response validity. Results indicate that the MRCI yields valid and reliable estimations of students' conceptual understanding of molecular randomness in the higher educational setting studied. Ultimately, the performance analysis sheds light on the extent and the limitations of students' understanding of the concept of stochasticity on a molecular level.

"The many faces of sorrow": An empirical exploration of the psychological plurality of sadness

Sadness has typically been associated with failure, defeat and loss, but it has also been suggested that sadness facilitates positive and restructuring emotional changes. This suggests that sadness is a multi-faceted emotion. This supports the idea that there might in fact be different facets of sadness that can be distinguished psychologically and physiologically. In the current set of studies, we explored this hypothesis. In a first stage, participants were asked to select sad emotional faces and scene stimuli either characterized or not by a key suggested sadness-related characteristic: loneliness or melancholy or misery or bereavement or despair. In a second stage, another set of participants was presented with the selected emotional faces and scene stimuli. They were assessed for differences in emotional, physiological and facial-expressive responses. The results showed that sad faces involving melancholy, misery, bereavement and despair were experienced as conferring dissociable physiological characteristics. Critical findings, in a final exploratory design, in a third stage, showed that a new set of participants could match emotional scenes to emotional faces with the same sadness-related characteristic with close to perfect precision performance. These findings suggest that melancholy, misery, bereavement and despair can be distinguishable emotional states associated with sadness.

How to Choose between Different Bayesian Posterior Indices for Hypothesis Testing in Practice

Hypothesis testing is an essential statistical method in experimental psychology and the cognitive sciences. The problems of traditional null hypothesis significance testing (NHST) have been discussed widely, and among the proposed solutions to the replication problems caused by the inappropriate use of significance tests and p-values is a shift toward Bayesian data analysis. However, Bayesian hypothesis testing is concerned with various posterior indices for significance and the size of an effect. This complicates Bayesian hypothesis testing in practice, as the availability of multiple Bayesian alternatives to the traditional p-value causes confusion which one to select and why. In this paper, various Bayesian posterior indices which have been proposed in the literature are compared and their benefits and limitations are discussed. The comparison shows that conceptually not all proposed Bayesian alternatives to NHST and p-values are beneficial, and the usefulness of some indices strongly depends on the study design and research goal. However, the comparison also reveals that there exist at least two candidates among the available Bayesian posterior indices which have appealing theoretical properties and are widely underused in the cognitive sciences.

The evidence interval and the Bayesian evidence value: On a unified theory for Bayesian hypothesis testing and interval estimation

Interval estimation is one of the most frequently used methods in statistical science, employed to provide a range of credible values a parameter is located in after taking into account the uncertainty in the data. However, while this interpretation only holds for Bayesian interval estimates, these suffer from two problems. First, Bayesian interval estimates can include values which have not been corroborated by observing the data. Second, Bayesian interval estimates and hypothesis tests can yield contradictory conclusions. In this paper a new theory for Bayesian hypothesis testing and interval estimation is presented. A new interval estimate is proposed, the Bayesian evidence interval, which is inspired by the Pereira-Stern theory of the full Bayesian significance test (FBST). It is shown that the evidence interval is a generalization of existing Bayesian interval estimates, that it solves the problems of standard Bayesian interval estimates and that it unifies Bayesian hypothesis testing and parameter estimation. The Bayesian evidence value is introduced, which quantifies the evidence for the (interval) null and alternative hypothesis. Based on the evidence interval and the evidence value, the (full) Bayesian evidence test (FBET) is proposed as a new, model-independent Bayesian hypothesis test. Additionally, a decision rule for hypothesis testing is derived which shows the relationship to a widely used decision rule based on the region of practical equivalence and Bayesian highest posterior density intervals and to the e-value in the FBST. In summary, the proposed method is universally applicable, computationally efficient, and while the evidence interval can be seen as an extension of existing Bayesian interval estimates, the FBET is a generalization of the FBST and contains it as a special case. Together, the theory developed provides a unification of Bayesian hypothesis testing and interval estimation and is made available in the R package fbst.

Bayesian identification of structural coefficients in causal models and the causal false-positive risk of confounders and colliders in linear Markovian models

Background

Causal inference has seen an increasing popularity in medical research. Estimation of causal effects from observational data allows to draw conclusions from data when randomized controlled trials cannot be conducted. Although the identification of structural causal models (SCM) and the calculation of structural coefficients has received much attention, a key requirement for valid causal inference is that conclusions are drawn based on the true data-generating model.

Methods

It remains widely unknown how large the probability is to reject the true structural causal model when observational data from it is sampled. The latter probability – the causal false-positive risk – is crucial, as rejection of the true causal model can induce bias in the estimation of causal effects. In this paper, the widely used causal models of confounders and colliders are studied regarding their causal false-positive risk in linear Markovian models. A simulation study is carried out which investigates the causal false-positive risk in Gaussian linear Markovian models. Therefore, the testable implications of the DAG corresponding to confounders and colliders are analyzed from a Bayesian perspective. Furthermore, the induced bias in estimating the structural coefficients and causal effects is studied.

Results

Results show that the false-positive risk of rejecting a true SCM of even simple building blocks like confounders and colliders is substantial. Importantly, estimation of average, direct and indirect causal effects can become strongly biased if a true model is rejected. The causal false-positive risk may thus serve as an indicator or proxy for the induced bias.

Conclusion

While the identification of structural coefficients and testable implications of causal models have been studied rigorously in the literature, this paper shows that causal inference also must develop new concepts for controlling the causal false-positive risk. Although a high risk cannot be equated with a substantial bias, it is indicative of the induced bias. The latter fact calls for the development of more advanced risk measures for committing a causal type I error in causal inference.

Supplementary Information

The online version contains supplementary material available at (10.1186/s12874-021-01473-w).

A New Bayesian Two-Sample t Test and Solution to the Behrens–Fisher Problem Based on Gaussian Mixture Modelling with Known Allocations

Testing differences between a treatment and control group is common practice in biomedical research like randomized controlled trials (RCT). The standard two-sample t test relies on null hypothesis significance testing (NHST) via p values, which has several drawbacks. Bayesian alternatives were recently introduced using the Bayes factor, which has its own limitations. This paper introduces an alternative to current Bayesian two-sample t tests by interpreting the underlying model as a two-component Gaussian mixture in which the effect size is the quantity of interest, which is most relevant in clinical research. Unlike p values or the Bayes factor, the proposed method focusses on estimation under uncertainty instead of explicit hypothesis testing. Therefore, via a Gibbs sampler, the posterior of the effect size is produced, which is used subsequently for either estimation under uncertainty or explicit hypothesis testing based on the region of practical equivalence (ROPE). An illustrative example, theoretical results and a simulation study show the usefulness of the proposed method, and the test is made available in the R package . In sum, the new Bayesian two-sample t test provides a solution to the Behrens–Fisher problem based on Gaussian mixture modelling.