Equivalence tests – A review

Use of wound infusion catheters for delivery of local anesthetic following standing partial ostectomy of thoracolumbar vertebral spinous processes in horses is not associated with increased surgical site infections

Background

Wound infusion catheters (WICs) have been used in humans and some veterinary species for post-operative local anesthetic administration following a variety of surgical procedures, aiming to reduce post-operative analgesia requirements and improve patient comfort. Benefit in reduction in pain, post-operative analgesia requirements and length of hospital stay are well documented in humans, but use of WICs may not have been widely adopted in veterinary species due to the concern of increased complications, such as dehiscence or surgical site infection (SSI), creating a barrier to their use. This study aimed to evaluate the use of WICs in horses undergoing standing partial ostectomy surgeries, document complications and investigate if the incidence of SSI was equivalent between those horses that did and did not have a WIC.

Methods

Clinical records were searched between January 2010-December 2023 for horses undergoing standing partial ostectomy surgery of thoracolumbar vertebral spinous processes at one institution. Population variables (age, breed, bodyweight), placement of a WIC or not, post-operative complications, analgesia protocols and surgical time were recorded. Horses received up to 0.1 mg kg-1 bupivacaine (0.5 mg mL-1) every 6-8 h via the WIC where one was placed. To compare SSI complication incidence between using or not using a WIC, a proportional independent equivalence test was used.

Results

There were 64 horses included in the final analysis with a WIC placed in 29/64 horses (45.3%) and 35/64 (54.7%) having no WIC placed at surgery. Incidence of short-term SSI was 11.4% (no WIC used) and 13.8% (WIC used), respectively. The difference in proportion of SSI between the presence or absence of a WIC was not significant [-0.024 (90% CI -0.181; 0.133); p = 0.94].

Conclusion

The incidence of SSIs was equivalent between groups whether a WIC was used or not. WICs should be considered as part of a multi-modal analgesic approach in the post-operative period. Further research into local anesthetic dosing and its impact on rescue analgesia requirements and pain-scores is warranted.

Metacognition and polyregulation in daily self-control conflicts

Metacognition is important for self-regulated learning, and it has recently been argued that it may play an important role in self-control more generally. We studied multiple aspects of metacognition in self-control, namely metacognitive knowledge including a person's repertoire ("toolbox") of different self-regulatory strategies, metacognitive regulation (planning, monitoring, and evaluation), and polyregulation (using more self-regulatory strategies in a single self-control conflict) as predictors of people's self-control success in daily life. In a preregistered experience sampling study, N = 503 participants reported their self-control conflicts up to eight times per day for 10 days, yielding 9,639 reports of daily self-control conflicts. Analyses showed that higher levels of metacognitive knowledge, planning, monitoring, evaluation, and polyregulation as well as a larger strategy repertoire were associated with higher levels of success in resolving daily self-control conflicts. Additionally, higher levels of trait self-control were associated with higher levels of metacognitive knowledge, planning, and monitoring. These findings highlight the importance of metacognition and polyregulation for successful self-control.

Finite sample corrections for average equivalence testing

Average (bio)equivalence tests are used to assess if a parameter, like the mean difference in treatment response between two conditions for example, lies within a given equivalence interval, hence allowing to conclude that the conditions have "equivalent" means. The two one-sided tests (TOST) procedure, consisting in testing whether the target parameter is respectively significantly greater and lower than some pre-defined lower and upper equivalence limits, is typically used in this context, usually by checking whether the confidence interval for the target parameter lies within these limits. This intuitive and visual procedure is however known to be conservative, especially in the case of highly variable drugs, where it shows a rapid power loss, often reaching zero, hence making it impossible to conclude for equivalence when it is actually true. Here, we propose a finite sample correction of the TOST procedure, theα $$ \alpha $$ -TOST, which consists in a correction of the significance level of the TOST allowing to guarantee a test size (or type-I error rate) ofα $$ \alpha $$ . This new procedure essentially corresponds to a finite sample and variability correction of the TOST procedure. We show that this procedure is uniformly more powerful than the TOST, easy to compute, and that its operating characteristics outperform the ones of its competitors. A case study about econazole nitrate deposition in porcine skin is used to illustrate the benefits of the proposed method and its advantages compared to other available procedures.

baymedr: an R package and web application for the calculation of Bayes factors for superiority, equivalence, and non-inferiority designs

BACKGROUND

Clinical trials often seek to determine the superiority, equivalence, or non-inferiority of an experimental condition (e.g., a new drug) compared to a control condition (e.g., a placebo or an already existing drug). The use of frequentist statistical methods to analyze data for these types of designs is ubiquitous even though they have several limitations. Bayesian inference remedies many of these shortcomings and allows for intuitive interpretations, but are currently difficult to implement for the applied researcher.

RESULTS

We outline the frequentist conceptualization of superiority, equivalence, and non-inferiority designs and discuss its disadvantages. Subsequently, we explain how Bayes factors can be used to compare the relative plausibility of competing hypotheses. We present baymedr, an R package and web application, that provides user-friendly tools for the computation of Bayes factors for superiority, equivalence, and non-inferiority designs. Instructions on how to use baymedr are provided and an example illustrates how existing results can be reanalyzed with baymedr.

CONCLUSIONS

Our baymedr R package and web application enable researchers to conduct Bayesian superiority, equivalence, and non-inferiority tests. baymedr is characterized by a user-friendly implementation, making it convenient for researchers who are not statistical experts. Using baymedr, it is possible to calculate Bayes factors based on raw data and summary statistics.

Comparing Literature- and Subreddit-Derived Laboratory Values in Polycystic Ovary Syndrome (PCOS): Validation of Clinical Data Posted on PCOS Reddit Forums

BACKGROUND

Polycystic ovary syndrome (PCOS) is a heterogeneous condition that affects 4% to 21% of people with ovaries. Inaccessibility or dissatisfaction with clinical treatment for PCOS has led to some individuals with the condition discussing their experiences in specialized web-based forums.

OBJECTIVE

This study explores the feasibility of using such web-based forums for clinical research purposes by gathering and analyzing laboratory test results posted in an active PCOS forum, specifically the PCOS subreddit hosted on Reddit.

METHODS

We gathered around 45,000 posts from the PCOS subreddit. A random subset of 5000 posts was manually read, and the presence of laboratory test results was labeled. These labeled posts were used to train a machine learning model to identify which of the remaining posts contained laboratory results. The laboratory results were extracted manually from the identified posts. These self-reported laboratory test results were compared with values in the published literature to assess whether the results were concordant with researcher-published values for PCOS cohorts. A total of 10 papers were chosen to represent published PCOS literature, with selection criteria including the Rotterdam diagnostic criteria for PCOS, a publication date within the last 20 years, and at least 50 participants with PCOS.

RESULTS

Overall, the general trends observed in the laboratory test results from the PCOS web-based forum were consistent with clinically reported PCOS. A number of results, such as follicle stimulating hormone, fasting insulin, and anti-Mullerian hormone, were concordant with published values for patients with PCOS. The high consistency of these results among the literature and when compared to the subreddit suggests that follicle stimulating hormone, fasting insulin, and anti-Mullerian hormone are more consistent across PCOS phenotypes than other test results. Some results, such as testosterone, sex hormone-binding globulin, and homeostasis model assessment-estimated insulin resistance index, were between those of PCOS literature values and normal values, as defined by clinical testing limits. Interestingly, other results, including dehydroepiandrosterone sulfate, luteinizing hormone, and fasting glucose, appeared to be slightly more dysregulated than those reported in the literature.

CONCLUSIONS

The differences between the forum-posted results and those published in the literature may be due to the selection process in clinical studies and the possibility that the forum disproportionally describes PCOS phenotypes that are less likely to be alleviated with medical intervention. However, the degree of concordance in most laboratory test values implied that the PCOS web-based forum participants were representative of research-identified PCOS cohorts. This validation of the PCOS subreddit grants the possibility for more research into the contents of the subreddit and the idea of undertaking similar research using the contents of other medical internet forums.

Relative uncertainty-based Bayesian interlaboratory consensus building

An interlaboratory comparison is typically conducted among the laboratories for the purpose of providing quality assurance and control. To solve the interlaboratory agreement problem, a distinct type of metrological challenge, a new uncertainty-based Bayesian strategy was developed and tested among environmental laboratories. A holistic algorithm with the key phases of sampling, outlier analysis, recognition, and simulation-based structure identification was developed and is being addressed in place of conventional indices and plots. Computer simulations showed that the proposed hybrid approach has no discernible sensitivity to outliers and that the agreement structure is transparent and robust. Some meta-data is also generated by the analysis based on relative uncertainty. To measure the performance and capability of Bayesian consensus building algorithm, the uncertainty intervals were established and comparative evaluations have been carried out using the conventional techniques. As a result, the suggested algorithm can explore both the laboratory performances (harmony) and the conformity between two independent samples. The algorithmic procedure features a generalizable framework that may be adapted in other fields to obtain a consensus among the laboratories.

Mindful self-focus-an interaction affecting Theory of Mind?

Is thinking about oneself helpful or harmful for understanding other people? The answer might depend on how a person thinks about themself. Mindfulness is one prominent construct that seems to affect the quality and content of a person's thoughts about themselves in the world. Thus, we hypothesize that the relationship between self-focus and Theory of Mind (ToM) is moderated by mindfulness. We evaluate our hypothesis with a large cross-sectional dataset (N = 543) of native and non-native German and English speakers using OLS and MM-estimated robust multiple regression analysis. We found a small but robust self-focus × mindfulness interaction effect on ToM so that there was a significant positive relation between self-focus and ToM for more mindful individuals and no significant relation for less mindful individuals. The findings support our hypothesis that mindfulness moderates the relationship between self-focus and ToM performance. We discuss the limitations and differences between the present study and previous findings.

Assessing individual equivalence in parallel group and crossover designs: Exact test and sample size procedures

The consideration of individual equivalence provides an essential alternative to average equivalence in two-group comparative studies. A common procedure for declaring individual equivalence adopts the tolerance intervals of the designated proportions of measurement differences. This statistical practice is a direct generalization of the widely used two one-sided tests (TOST) for average equivalence. Such TOST extensions often do not have adequate control of Type I error and result in excessively conservative tests. To signify and resolve the underlying issues of existing methods, this paper presents exact tests for assessing individual equivalence between two treatments under parallel group and crossover designs. Rigorous evaluations are conducted to clarify the discrepancy of critical values and Type I error probabilities between the equivalence procedures. The findings elucidate the shortcoming of the TOST technique and the advantage of the proposed approach. The associated power and sample size calculations are also justified through simulation studies.

Benchmark dose modelling in regulatory ecotoxicology, a potential tool in pest management

For several authorities, benchmark dose (BMD) methodology has become the recommended approach by which to derive reference values for risk assessment. However, in practice, the BMD approach is not standard use in risk assessment for pesticides where the no observed adverse effect level, lowest observed adverse effect level and effective dose (ED₅₀ or ED_x ) prevail. Regression-based BMD and the benchmark dose lower confidence limit (BMDL) have several advantages, such as utilizing more information from the generated data and being less dependent on tested dose levels. However, the BMD approach requires some degree of expert knowledge for defining an appropriate risk level for estimating the BMD and using more sophisticated statistical methods to calculate BMD and BMDL. The BMD approach is one way to move away from p value-based binary decision-making towards putting the weight on effect sizes. We review the advantages and disadvantages of focusing on the BMD approach for risk assessment of pesticides. Further, we discuss potential applications in efficacy trials for pest management purposes. © 2021 Society of Chemical Industry.

All-or-none neural mechanisms underlying face categorization: evidence from the N170

Categorization of visual stimuli is an intrinsic aspect of human perception. Whether the cortical mechanisms underlying categorization operate in an all-or-none or graded fashion remains unclear. In this study, we addressed this issue in the context of the face-specific N170. Specifically, we investigated whether N170 amplitudes grade with the amount of face information available in an image, or a full response is generated whenever a face is perceived. We employed linear mixed-effects modeling to inspect the dependency of N170 amplitudes on stimulus properties and duration, and their relationships to participants' subjective perception. Consistent with previous studies, we found a stronger N170 evoked by faces presented for longer durations. However, further analysis with equivalence tests revealed that this duration effect was eliminated when only faces perceived with high confidence were considered. Therefore, previous evidence supporting the graded hypothesis is more likely to be an artifact of mixing heterogeneous "all" and "none" trial types in signal averaging. These results support the hypothesis that the N170 is generated in an all-or-none manner and, by extension, suggest that categorization of faces may follow a similar pattern.

Myths and Methodologies: The use of equivalence and non-inferiority tests for interventional studies in exercise physiology and sport science

NEW FINDINGS

What is the topic of this review? The traditional null-hypothesis test allows making informed decisions about whether an experimental intervention is superior to a control while controlling the Type I and Type II error rates. Equivalence and non-inferiority tests allow determining whether two interventions are similar in efficacy or whether one is not unacceptably worse than the other. What advances does it highlight? Equivalence and non-inferiority designs may better fit the research questions of exercise physiologists and sport scientists when investigating new interventions that have better cost-effectiveness, are safer and easier to implement, or are less demanding than the standard.

ABSTRACT

Exercise physiology and sport science have traditionally made use of the null hypothesis of no difference to make decisions about experimental interventions. This article aims to review current statistical approaches typically used by exercise physiologists and sport scientists for the design and analysis of experimental interventions and to highlight the importance of including equivalence and non-inferiority studies, which address different research questions than deciding whether an effect is present. Firstly, we briefly describe the most common approaches, along with their rationale, to investigate the effects of different interventions. We then discuss the main steps involved in the design and analysis of equivalence and non-inferiority studies, commonly performed in other research fields, with worked examples from exercise physiology and sport science scenarios. Finally, we provide recommendations to exercise physiologists and sport scientists who would like to apply the different approaches in future research. We hope this work will promote the correct use of equivalence and non-inferiority designs in exercise physiology and sport science whenever the research context, conditions, applications, researchers' interests, or reasonable beliefs, justify these approaches. This article is protected by copyright. All rights reserved.

Providing Evidence for the Null Hypothesis in Functional Magnetic Resonance Imaging Using Group-Level Bayesian Inference

Classical null hypothesis significance testing is limited to the rejection of the point-null hypothesis; it does not allow the interpretation of non-significant results. This leads to a bias against the null hypothesis. Herein, we discuss statistical approaches to ‘null effect’ assessment focusing on the Bayesian parameter inference (BPI). Although Bayesian methods have been theoretically elaborated and implemented in common neuroimaging software packages, they are not widely used for ‘null effect’ assessment. BPI considers the posterior probability of finding the effect within or outside the region of practical equivalence to the null value. It can be used to find both ‘activated/deactivated’ and ‘not activated’ voxels or to indicate that the obtained data are not sufficient using a single decision rule. It also allows to evaluate the data as the sample size increases and decide to stop the experiment if the obtained data are sufficient to make a confident inference. To demonstrate the advantages of using BPI for fMRI data group analysis, we compare it with classical null hypothesis significance testing on empirical data. We also use simulated data to show how BPI performs under different effect sizes, noise levels, noise distributions and sample sizes. Finally, we consider the problem of defining the region of practical equivalence for BPI and discuss possible applications of BPI in fMRI studies. To facilitate ‘null effect’ assessment for fMRI practitioners, we provide Statistical Parametric Mapping 12 based toolbox for Bayesian inference.

A robust method for the assessment of average bioequivalence in the presence of outliers and skewness

PURPOSE

In this paper, we propose a robust Bayesian method for the assessment of average bioequivalence based on data from conventional crossover studies. We evaluate and motivate empirically the need for robust methods in bioequivalence studies by comparing the results of robust and conventional statistical methods in a large data pool of bioequivalence studies.

METHODS

Robustness of the statistical methodology is achieved by replacing the normal distributions for residuals in the linear mixed model with skew-t distributions. In this way, the statistical model can accommodate skew and heavy-tailed data, particularly outliers, yielding robust statistical inference without the need for excluding outliers from the analysis. We performed a simulation study to investigate and compare the performance of the robust and conventional models.

RESULTS

Our study shows that in some trials, the distribution of residuals is skew and heavy-tailed. In the presence of outliers, the 90% confidence intervals for the ratio of geometric means tend to be narrower for the robust methods than for the conventional method. Our simulation study shows that the robust method has suitable frequentist properties and yields more precise confidence intervals and higher statistical power than the conventional maximum likelihood method when outliers are present in the data.

CONCLUSIONS

As a sensitivity analysis, we recommend the fit of robust models for handling outliers that are occasionally encountered in crossover design bioequivalence data.

Gamma-irradiation of common biological samples for stable carbon and nitrogen isotope and elemental analyses

RATIONALE

Around the world biosecurity measures are being strengthened to prevent the spread of pests and diseases across national and international borders. Quarantine protocols that involve sample sterilisation have potential effects on sample integrity. The consequences of sterilisation methods such as gamma (γ)-irradiation on the elemental and chemical properties of biological samples have not been widely examined.

METHODS

We tested the effect of γ-irradiation (50 kGy) on the stable carbon and nitrogen isotope compositions (δ¹³ C and δ¹⁵ N values) and elemental concentrations (C % and N %) of common biological samples (fish, plants and bulk soils). The analysis used a continuous flow system consisting of a Delta V Plus isotope ratio mass spectrometer connected with a Thermo Flash 1112 elemental analyser via a ConFlo IV interface. Results were compared using two one-sided tests (TOST) to test for statistical similarity between paired samples.

RESULTS

There was no change in the δ¹⁵ N values or N % of γ-irradiated samples, and only small changes to the δ¹³ C values of consumers (range: 0.01‰ to 0.04‰), producers (-0.02‰ to 0.04‰) and sediments (0‰ to 0.07‰). The magnitude of change in δ¹³ C values was greatest at low carbon concentrations but appeared negligible when measured against replicated sample analysis and the combined analytical uncertainty (i.e., 0.10‰). The C % values of irradiated samples were higher for consumers (0.23%) and lower for producers and sediments (0.04% and 0.05%, respectively) which may have implications for certain types of biological material.

CONCLUSIONS

Routine γ-irradiation has little effect on the stable carbon and nitrogen isotope compositions of common biological samples and marginal effects on carbon elemental concentrations. This is unlikely to warrant concerns since the observed difference is typically of a magnitude lower than other sources of potential uncertainty.

Bayesian Hodges-Lehmann tests for statistical equivalence in the two-sample setting: Power analysis, type I error rates and equivalence boundary selection in biomedical research

BACKGROUND

Null hypothesis significance testing (NHST) is among the most frequently employed methods in the biomedical sciences. However, the problems of NHST and p-values have been discussed widely and various Bayesian alternatives have been proposed. Some proposals focus on equivalence testing, which aims at testing an interval hypothesis instead of a precise hypothesis. An interval hypothesis includes a small range of parameter values instead of a single null value and the idea goes back to Hodges and Lehmann. As researchers can always expect to observe some (although often negligibly small) effect size, interval hypotheses are more realistic for biomedical research. However, the selection of an equivalence region (the interval boundaries) often seems arbitrary and several Bayesian approaches to equivalence testing coexist.

METHODS

A new proposal is made how to determine the equivalence region for Bayesian equivalence tests based on objective criteria like type I error rate and power. Existing approaches to Bayesian equivalence testing in the two-sample setting are discussed with a focus on the Bayes factor and the region of practical equivalence (ROPE). A simulation study derives the necessary results to make use of the new method in the two-sample setting, which is among the most frequently carried out procedures in biomedical research.

RESULTS

Bayesian Hodges-Lehmann tests for statistical equivalence differ in their sensitivity to the prior modeling, power, and the associated type I error rates. The relationship between type I error rates, power and sample sizes for existing Bayesian equivalence tests is identified in the two-sample setting. Results allow to determine the equivalence region based on the new method by incorporating such objective criteria. Importantly, results show that not only can prior selection influence the type I error rate and power, but the relationship is even reverse for the Bayes factor and ROPE based equivalence tests.

CONCLUSION

Based on the results, researchers can select between the existing Bayesian Hodges-Lehmann tests for statistical equivalence and determine the equivalence region based on objective criteria, thus improving the reproducibility of biomedical research.

Using historical control data in bioassays for regulatory toxicology

Historical control data (HCD) consist of pooled control group responses from bioassays. These data must be collected and are often used or reported in regulatory toxicology studies for multiple purposes: as quality assurance for the test system, to help identify toxicological effects and their effect-size relevance and to address the statistical multiple comparison problem. The current manuscript reviews the various classical and potential new approaches for using HCD. Issues in current practice are identified and recommendations for improved use and discussion are provided. Furthermore, stakeholders are invited to discuss whether it is necessary to consider uncertainty when using HCD formally and statistically in toxicological discussions and whether binary inclusion/exclusion criteria for HCD should be revised to a tiered information contribution to assessments. Overall, the critical value of HCD in toxicological bioassays is highlighted when used in a weight-of-evidence assessment.

Improved procedures and computer programs for equivalence assessment of correlation coefficients

The correlation coefficient is the most commonly used measure for summarizing the magnitude and direction of linear relationship between two response variables. Considerable literature has been devoted to the inference procedures for significance tests and confidence intervals of correlations. However, the essential problem of evaluating correlation equivalence has not been adequately examined. For the purpose of expanding the usefulness of correlational techniques, this article focuses on the Pearson product-moment correlation coefficient and the Fisher's z transformation for developing equivalence procedures of correlation coefficients. Equivalence tests are proposed to assess whether a correlation coefficient is within a designated reference range for declaring equivalence decisions. The important aspects of Type I error rate, power calculation, and sample size determination are also considered. Special emphasis is given to clarify the nature and deficiency of the two one-sided tests for detecting a lack of association. The findings demonstrate the inappropriateness of existing methods for equivalence appraisal and validate the suggested techniques as reliable and primary tools in correlation analysis.

Is Absolute Pitch Associated With Musical Tension Processing?

Absolute pitch (AP) is a superior ability to identify or produce musical tones without a reference tone. Although a few studies have investigated the relationship between AP and high-level music processing such as tonality and syntactic processing, very little is known about whether AP is related to musical tension processing. To address this issue, 20 AP possessors and 20 matched non-AP possessors listened to major and minor melodies and rated the levels of perceived and felt musical tension using a continuous response digital interface dial. Results indicated that the major melodies were perceived and felt as less tense than the minor ones by AP and non-AP possessors. However, there was weak evidence for no differences between AP and non-AP possessors in the perception and experience of musical tension, suggesting that AP may be independent of the processing of musical tension. The implications of these findings are discussed.

The Validity, Time Burden, and User Satisfaction of the FoodImage™ Smartphone App for Food Waste Measurement Versus Diaries: A Randomized Crossover Trial

The FoodImage™ smartphone app transmits to researchers users' photographs of food selection and food waste, and includes user-tagged information about waste reasons and destination. Twenty-four participants were trained to record food waste using FoodImage, food waste diaries requiring visual estimation of waste quantities, and diaries requiring scale weights. Participants used each method during three staged food-waste scenarios (food preparation, eating, and clean-out) in a randomized crossover trial. Two participants had extreme values for the weighed diary method; therefore, accuracy results are reported with and without these two participants' data. Error was calculated as waste estimated with the experimental method minus directly weighed waste. Mean absolute error from FoodImage was significantly smaller than or equal to the error from both diary methods in each scenario. Furthermore, the mean values from FoodImage were equivalent to directly weighed values in two out of the three tasks; while weighed diaries were equivalent in two tasks only when the two participants with extreme values were removed. Visually estimated diaries were equivalent for only one task. All 24 participants preferred FoodImage to diaries and all rated FoodImage as less time consuming. Over one week, FoodImage would require ~24 fewer minutes of users' time to record all data. Unlike food waste diaries, FoodImage also transmits data to researchers in real-time and provides detailed data on food selection and intake. ClinicalTrials.gov Identifier: NCT03309306.

P > .05: The incorrect interpretation of "not significant" results is a significant problem

Statistically nonsignificant (p > .05) results from a null hypothesis significance test (NHST) are often mistakenly interpreted as evidence that the null hypothesis is true-that there is "no effect" or "no difference." However, many of these results occur because the study had low statistical power to detect an effect. Power below 50% is common, in which case a result of no statistical significance is more likely to be incorrect than correct. The inference of "no effect" is not valid even if power is high. NHST assumes that the null hypothesis is true; p is the probability of the data under the assumption that there is no effect. A statistical test cannot confirm what it assumes. These incorrect statistical inferences could be eliminated if decisions based on p values were replaced by a biological evaluation of effect sizes and their confidence intervals. For a single study, the observed effect size is the best estimate of the population effect size, regardless of the p value. Unlike p values, confidence intervals provide information about the precision of the observed effect. In the biomedical and pharmacology literature, methods have been developed to evaluate whether effects are "equivalent," rather than zero, as tested with NHST. These methods could be used by biological anthropologists to evaluate the presence or absence of meaningful biological effects. Most of what appears to be known about no difference or no effect between sexes, between populations, between treatments, and other circumstances in the biological anthropology literature is based on invalid statistical inference.

Comparison of alternative approaches for difference, noninferiority, and equivalence testing of normal percentiles

Background

Percentiles are widely used in scientific research for determining the comparative magnitude and reference limit of quantitative measurements. The investigations for point and interval estimation of normal percentiles are well documented in the literature. However, the corresponding statistical tests of hypothesis have received relatively little attention.

Methods

To facilitate data analysis and design planning of percentile study, this paper aims to present hypothesis testing procedures and associated power functions for assessing the difference, noninferiority, and equivalence of normal percentiles.

Results

Numerical illustrations about drug dissolution are provided to demonstrate the usefulness of the suggested exact approaches and the deficiency of approximate methods.

Conclusions

The exact approaches are superior to the approximate methods on the basis of control of Type I errors. Computer algorithms are constructed to implement the recommended test procedures and sample size calculations for percentile analysis.

Optimal Sample Sizes for Testing the Equivalence of Two Means

Abstract. Equivalence tests (also known as similarity or parity tests) have become more and more popular in addition to equality tests. However, in testing the equivalence of two population means, approximate sample sizes developed using conventional techniques found in the literature on this topic have usually been under-valued as having less statistical power than is required. In this paper, the authors first address the reason for this problem and then provide a solution using an exhaustive local search algorithm to find the optimal sample size. The proposed method is not only accurate but is also flexible so that unequal variances or sampling unit costs for different groups can be considered using different sample size allocations. Figures and a numerical example are presented to demonstrate various configurations. An R Shiny App is also available for easy use ( https://optimal-sample-size.shinyapps.io/equivalence-of-means/ ).

Equivalence analysis to support environmental safety assessment: Using nontarget organism count data from field trials with cisgenically modified potato

This paper considers the statistical analysis of entomological count data from field experiments with genetically modified (GM) plants. Such trials are carried out to assess environmental safety. Potential effects on nontarget organisms (NTOs), as indicators of biodiversity, are investigated. The European Food Safety Authority (EFSA) gives broad guidance on the environmental risk assessment (ERA) of GM plants. Field experiments must contain suitable comparator crops as a benchmark for the assessment of designated endpoints. In this paper, a detailed protocol is proposed to perform data analysis for the purpose of assessing environmental safety. The protocol includes the specification of a list of endpoints and their hierarchical relations, the specification of intended levels of data analysis, and the specification of provisional limits of concern to decide on the need for further investigation. The protocol emphasizes a graphical representation of estimates and confidence intervals for the ratio of mean abundances for the GM plant and its comparator crop. Interpretation relies mainly on equivalence testing in which confidence intervals are compared with the limits of concern. The proposed methodology is illustrated with entomological count data resulting from multiyear, multilocation field trials. A cisgenically modified potato line (with enhanced resistance to late blight disease) was compared to the original conventional potato variety in the Netherlands and Ireland in two successive years (2013, 2014). It is shown that the protocol encompasses alternative schemes for safety assessment resulting from different research questions and/or expert choices. Graphical displays of equivalence testing at several hierarchical levels and their interpretation are presented for one of these schemes. The proposed approaches should be of help in the ERA of GM or other novel plants.

Sustainable use of tainted boar meat: Blending is a strategy for processed products

While forming mixtures is a widely used approach for other raw materials in food industry, it has not yet been systematically analyzed for boar tainted meat. That is why we simultaneously studied four factors relevant for the production of emulsion-type sausages: percentage boar meat (skatole concentrations up to 0.3 μg/g, androstenone up to 3.8 μg/g in melted backfat), duration of traditional smoke and concentration levels of two spices. 16 variants of Frankfurters were produced in two independent studies and evaluated by in total 211 consumers. A linear mixed effects model revealed that increased levels of boar tainted meat significantly reduced consumer acceptance which could not be compensated by increased smoke or spice levels. We propose a non-inferiority test to identify the mixture which is similarly accepted as the reference made without boar tainted meat. Up to 33% tainted boar meat is proposed, assuming a liking drop of 0.5 on a 9 point liking scale as benchmark for an inferior product.

Improving Inferences About Null Effects With Bayes Factors and Equivalence Tests

Researchers often conclude an effect is absent when a null-hypothesis significance test yields a nonsignificant p value. However, it is neither logically nor statistically correct to conclude an effect is absent when a hypothesis test is not significant. We present two methods to evaluate the presence or absence of effects: Equivalence testing (based on frequentist statistics) and Bayes factors (based on Bayesian statistics). In four examples from the gerontology literature, we illustrate different ways to specify alternative models that can be used to reject the presence of a meaningful or predicted effect in hypothesis tests. We provide detailed explanations of how to calculate, report, and interpret Bayes factors and equivalence tests. We also discuss how to design informative studies that can provide support for a null model or for the absence of a meaningful effect. The conceptual differences between Bayes factors and equivalence tests are discussed, and we also note when and why they might lead to similar or different inferences in practice. It is important that researchers are able to falsify predictions or can quantify the support for predicted null effects. Bayes factors and equivalence tests provide useful statistical tools to improve inferences about null effects.

Equivalence Testing for Psychological Research: A Tutorial

Psychologists must be able to test both for the presence of an effect and for the absence of an effect. In addition to testing against zero, researchers can use the two one-sided tests (TOST) procedure to test for equivalence and reject the presence of a smallest effect size of interest (SESOI). The TOST procedure can be used to determine if an observed effect is surprisingly small, given that a true effect at least as extreme as the SESOI exists. We explain a range of approaches to determine the SESOI in psychological science and provide detailed examples of how equivalence tests should be performed and reported. Equivalence tests are an important extension of the statistical tools psychologists currently use and enable researchers to falsify predictions about the presence, and declare the absence, of meaningful effects.

Conditional equivalence testing: An alternative remedy for publication bias

We introduce a publication policy that incorporates "conditional equivalence testing" (CET), a two-stage testing scheme in which standard NHST is followed conditionally by testing for equivalence. The idea of CET is carefully considered as it has the potential to address recent concerns about reproducibility and the limited publication of null results. In this paper we detail the implementation of CET, investigate similarities with a Bayesian testing scheme, and outline the basis for how a scientific journal could proceed to reduce publication bias while remaining relevant.

Analysis of methods commonly used in biomedicine for treatment versus control comparison of very small samples

BACKGROUND AND OBJECTIVE

A rough estimate indicated that use of samples of size not larger than ten is not uncommon in biomedical research and that many of such studies are limited to strong effects due to sample sizes smaller than six. For data collected from biomedical experiments it is also often unknown if mathematical requirements incorporated in the sample comparison methods are satisfied.

METHODS

Computer simulated experiments were used to examine performance of methods for qualitative sample comparison and its dependence on the effectiveness of exposure, effect intensity, distribution of studied parameter values in the population, and sample size. The Type I and Type II errors, their average, as well as the maximal errors were considered.

RESULTS

The sample size 9 and the t-test method with p = 5% ensured error smaller than 5% even for weak effects. For sample sizes 6-8 the same method enabled detection of weak effects with errors smaller than 20%. If the sample sizes were 3-5, weak effects could not be detected with an acceptable error; however, the smallest maximal error in the most general case that includes weak effects is granted by the standard error of the mean method. The increase of sample size from 5 to 9 led to seven times more accurate detection of weak effects. Strong effects were detected regardless of the sample size and method used.

CONCLUSIONS

The minimal recommended sample size for biomedical experiments is 9. Use of smaller sizes and the method of their comparison should be justified by the objective of the experiment.

Genetic alterations driving metastatic colony formation are acquired outside of the primary tumour in melanoma

Mouse models indicate that metastatic dissemination occurs extremely early; however, the timing in human cancers is unknown. We therefore determined the time point of metastatic seeding relative to tumour thickness and genomic alterations in melanoma. Here, we find that lymphatic dissemination occurs shortly after dermal invasion of the primary lesion at a median thickness of ~0.5 mm and that typical driver changes, including BRAF mutation and gained or lost regions comprising genes like MET or CDKNA2, are acquired within the lymph node at the time of colony formation. These changes define a colonisation signature that was linked to xenograft formation in immunodeficient mice and death from melanoma. Thus, melanoma cells leave primary tumours early and evolve at different sites in parallel. We propose a model of metastatic melanoma dormancy, evolution and colonisation that will inform direct monitoring of adjuvant therapy targets.

Equivalence Tests: A Practical Primer for t Tests, Correlations, and Meta-Analyses

Scientists should be able to provide support for the absence of a meaningful effect. Currently, researchers often incorrectly conclude an effect is absent based a nonsignificant result. A widely recommended approach within a frequentist framework is to test for equivalence. In equivalence tests, such as the two one-sided tests (TOST) procedure discussed in this article, an upper and lower equivalence bound is specified based on the smallest effect size of interest. The TOST procedure can be used to statistically reject the presence of effects large enough to be considered worthwhile. This practical primer with accompanying spreadsheet and R package enables psychologists to easily perform equivalence tests (and power analyses) by setting equivalence bounds based on standardized effect sizes and provides recommendations to prespecify equivalence bounds. Extending your statistical tool kit with equivalence tests is an easy way to improve your statistical and theoretical inferences.

Exact Power and Sample Size Calculations for the Two One-Sided Tests of Equivalence

Equivalent testing has been strongly recommended for demonstrating the comparability of treatment effects in a wide variety of research fields including medical studies. Although the essential properties of the favorable two one-sided tests of equivalence have been addressed in the literature, the associated power and sample size calculations were illustrated mainly for selecting the most appropriate approximate method. Moreover, conventional power analysis does not consider the allocation restrictions and cost issues of different sample size choices. To extend the practical usefulness of the two one-sided tests procedure, this article describes exact approaches to sample size determinations under various allocation and cost considerations. Because the presented features are not generally available in common software packages, both R and SAS computer codes are presented to implement the suggested power and sample size computations for planning equivalence studies. The exact power function of the TOST procedure is employed to compute optimal sample sizes under four design schemes allowing for different allocation and cost concerns. The proposed power and sample size methodology should be useful for medical sciences to plan equivalence studies.

A comparative appraisal of two equivalence tests for multiple standardized effects

Equivalence testing is recommended as a better alternative to the traditional difference-based methods for demonstrating the comparability of two or more treatment effects. Although equivalent tests of two groups are widely discussed, the natural extensions for assessing equivalence between several groups have not been well examined. This article provides a detailed and schematic comparison of the ANOVA F and the studentized range tests for evaluating the comparability of several standardized effects. Power and sample size appraisals of the two grossly distinct approaches are conducted in terms of a constraint on the range of the standardized means when the standard deviation of the standardized means is fixed. Although neither method is uniformly more powerful, the studentized range test has a clear advantage in sample size requirements necessary to achieve a given power when the underlying effect configurations are close to the priori minimum difference for determining equivalence. For actual application of equivalence tests and advance planning of equivalence studies, both SAS and R computer codes are available as supplementary files to implement the calculations of critical values, p-values, power levels, and sample sizes.

Impact of high hydrostatic pressure treatment on microbial communities in chinese water-boiled salted duck

The objective of this study was to determine the impact of high hydrostatic pressure (HHP) treatment on microbial communities in Chinese water-boiled salted duck (CWBSD) and the synergistic effect of HHP and mild heat treatment. In this work, the bacterial diversity was evaluated by using both a culture-dependent method and denaturing gradient gel electrophoresis. The total aerobic bacterial counts in pressure-treated samples were significantly lower than those in controls, which indicated that HHP could extend the shelf life of CWBSD. Weissella hellenica and Enterobacteriaceae, the predominant bacteria found in CWBSD, were not detected after HHP treatment. On the other hand, Staphylococcus spp. and Bacillus spp. became the predominant bacteria in HHP-treated samples. Moreover, the inhibitory effect was greater at 400 MPa than at 200 MPa combined with mild heat at 40°C. This study investigated the diversity of bacteria in HHP-treated CWBSD, and the information derived from this research may help to understand the bacterial ecology and develop effective HHP treatments to extend the shelf life of CWBSD.