What Have We (Not) Learnt from Millions of Scientific Papers with P Values?

Three Decades of Adolescent Health: Unveiling Global Trends Across 41 Countries in Psychological and Somatic Complaints (1994-2022)

Objectives

This study examined (non-)monotonic time trends in psychological and somatic complaints among adolescents, along with gender differences.

Methods

Repeated cross-sectional Health Behaviour in School-aged Children (HBSC) data from 1994 to 2022 covering 15-year-old adolescents from 41 countries (N = 470,797) were analysed. Three polynomial logistic regression models (linear, quadratic, cubic) were tested for best fit, including separate analyses by gender and health complaints dimension.

Results

Time trend patterns varied by gender and health complaints dimension. Increases were found in 82.3% of cases (linear 25%, quadratic U-shaped 28.7%, cubic 28.7%), while 14% showed no clear trend, and 3.7% decreased. Boys typically showed linear increases or no clear trend over time, whereas girls generally showed cubic or U-shaped trends. Psychological complaints often displayed U-shaped or cubic patterns, whereas somatic complaints mostly showed linear increases.

Conclusion

Psychological and somatic complaints demonstrated diverse time trend patterns across countries, with non-monotonic patterns (U-shaped and cubic) frequently observed alongside linear increases. These findings highlight the complexity of changes within countries over three decades, suggesting that linear modelling may not effectively capture this heterogeneity.

The relative effectiveness of a high-dose quadrivalent influenza vaccine versus standard-dose quadrivalent influenza vaccines in older adults in France: a retrospective cohort study during the 2021-2022 influenza season

OBJECTIVES

High-dose quadrivalent influenza vaccine (HD-QIV) was introduced during the 2021/2022 influenza season in France for adults aged ≥65 years as an alternative to standard-dose quadrivalent influenza vaccine (SD-QIV). The aim of this study is to estimate the relative vaccine effectiveness of HD-QIV vs. SD-QIV against influenza-related hospitalizations in France.

METHODS

Community-dwelling individuals aged ≥65 years with reimbursed influenza vaccine claims during the 2021/2022 influenza season were included in the French national health insurance database. Individuals were followed up from vaccination day to 30 June 2022, nursing home admission or death date. Baseline socio-demographic and health characteristics were identified from medical records over the five previous years. Hospitalizations for influenza and other causes were recorded from 14 days after vaccination until the end of follow-up. HD-QIV and SD-QIV vaccinees were matched using 1:4 propensity score matching with an exact constraint on age group, sex, week of vaccination, and region. Incidence rate ratios were estimated using zero-inflated Poisson or zero-inflated negative binomial regression models.

RESULTS

We matched 405 385 HD-QIV to 1 621 540 SD-QIV vaccinees. HD-QIV was associated with a 23.3% (95% CI, 8.4-35.8) lower rate of influenza hospitalizations compared with SD-QIV (69.5/100 000 person years vs. 90.5/100 000 person years). Post-matching, we observed higher rates in the HD-QIV group for hospitalizations non-specific to influenza and negative control outcomes, suggesting residual confounding by indication.

DISCUSSION

HD-QIV was associated with lower influenza-related hospitalization rates vs. SD-QIV, consistent with existing evidence, in the context of high SARS-CoV-2 circulation in France and likely prioritization of HD-QIV for older/more comorbid individuals.

Changing Landscape of Randomized Clinical Trials in Stroke: Explaining Contemporary Trial Designs and Methods

Evidence generated from randomized clinical trials (RCTs) plays an indispensable role in advancing clinical stroke care. Although the number of stroke-related RCTs published every year has grown exponentially over the past 25 years, the execution and completion of RCTs, particularly those conducted in a hyperacute setting, have grown more complicated and challenging over the years. In addition to the practical challenges associated with conducting a clinical trial, like obtaining human subjects approval, identifying clinical sites, training trial personnel, and enrolling the target number of patients within the available funding and timeline, the complexity of contemporary RCT designs and analyses has become much more exacting. It is no longer sufficient to have a decent understanding of the 2-arm, placebo-controlled RCT, combined with a rudimentary grasp of the P value; things are now much more complicated. Innovations in trial design and analysis, including adaptive, Bayesian, platform, and noninferiority designs, have occurred to address the problems of poor trial efficiency. However, these advances require the end user to have a much greater level of understanding regarding the rationale, conduct, analysis, and interpretation of each design. While these newer designs seek greater efficiency, there are inevitably tradeoffs that need to be understood. In this month's edition of Stroke, we introduce a new series designed to help fill in these knowledge gaps. Over the next few months, 4 papers will be published that address major design innovations (adaptive, Bayesian, platform, and noninferiority) with the aim of illustrating how these approaches can make trials more efficient (where efficiency is defined as getting to the right answer, sooner, with a potentially lower sample size). In addition to introducing this series, this current article also reviews traditional hypothesis testing and the common misinterpretations of the P value; fortunately, new philosophical schools of inference are beginning to vanquish the overreliance on the P value. We are excited about the opportunity to educate the Stroke readership about these new trial designs and the profound implications that they bring.

Beyond 'statistical significance': A nontechnical primer of Bayesian statistics and Bayes factors for health researchers

RATIONALE

Hypothesis testing is integral to health research and is commonly completed through frequentist statistics focused on computing p values. p Values have been long criticized for offering limited information about the relationship of variables and strength of evidence concerning the plausibility, presence and certainty of associations among variables. Bayesian statistics is a potential alternative for inference-making. Despite emerging discussion on Bayesian statistics across various disciplines, the uptake of Bayesian statistics in health research is still limited.

AIM

To offer a primer on Bayesian statistics and Bayes factors for health researchers to gain preliminary knowledge of its use, application and interpretation in health research.

METHODS

Theoretical and empirical literature on Bayesian statistics and methods were used to develop this methodological primer.

CONCLUSIONS

Using Bayesian statistics in health research without a careful and complete understanding of its underlying philosophy and differences from frequentist testing, estimation and interpretation methods can result in similar ritualistic use as done for p values.

IMPLICATIONS

Health researchers should supplement frequentists statistics with Bayesian statistics when analysing research data. The overreliance on p values for clinical decisions making should be avoided. Bayes factors offer a more intuitive measure of assessing the strength of evidence for null and alternative hypothesis.

Introducing Attribute Association Graphs to Facilitate Medical Data Exploration: Development and Evaluation Using Epidemiological Study Data

BACKGROUND

Interpretability and intuitive visualization facilitate medical knowledge generation through big data. In addition, robustness to high-dimensional and missing data is a requirement for statistical approaches in the medical domain. A method tailored to the needs of physicians must meet all the abovementioned criteria.

OBJECTIVE

This study aims to develop an accessible tool for visual data exploration without the need for programming knowledge, adjusting complex parameterizations, or handling missing data. We sought to use statistical analysis using the setting of disease and control cohorts familiar to clinical researchers. We aimed to guide the user by identifying and highlighting data patterns associated with disease and reveal relations between attributes within the data set.

METHODS

We introduce the attribute association graph, a novel graph structure designed for visual data exploration using robust statistical metrics. The nodes capture frequencies of participant attributes in disease and control cohorts as well as deviations between groups. The edges represent conditional relations between attributes. The graph is visualized using the Neo4j (Neo4j, Inc) data platform and can be interactively explored without the need for technical knowledge. Nodes with high deviations between cohorts and edges of noticeable conditional relationship are highlighted to guide the user during the exploration. The graph is accompanied by a dashboard visualizing variable distributions. For evaluation, we applied the graph and dashboard to the Hamburg City Health Study data set, a large cohort study conducted in the city of Hamburg, Germany. All data structures can be accessed freely by researchers, physicians, and patients. In addition, we developed a user test conducted with physicians incorporating the System Usability Scale, individual questions, and user tasks.

RESULTS

We evaluated the attribute association graph and dashboard through an exemplary data analysis of participants with a general cardiovascular disease in the Hamburg City Health Study data set. All results extracted from the graph structure and dashboard are in accordance with findings from the literature, except for unusually low cholesterol levels in participants with cardiovascular disease, which could be induced by medication. In addition, 95% CIs of Pearson correlation coefficients were calculated for all associations identified during the data analysis, confirming the results. In addition, a user test with 10 physicians assessing the usability of the proposed methods was conducted. A System Usability Scale score of 70.5% and average successful task completion of 81.4% were reported.

CONCLUSIONS

The proposed attribute association graph and dashboard enable intuitive visual data exploration. They are robust to high-dimensional as well as missing data and require no parameterization. The usability for clinicians was confirmed via a user test, and the validity of the statistical results was confirmed by associations known from literature and standard statistical inference.

Interplay between drought and plant viruses co-infecting melon plants

Drought affects crops directly, and indirectly by affecting the activity of insect pests and the transmitted pathogens. Here, we established an experiment with well-watered or water-stressed melon plants, later single infected with either cucumber mosaic virus (CMV: non-persistent), or cucurbit aphid-borne yellow virus (CABYV: persistent), or both CMV and CABYV, and mock-inoculated control. We tested whether i) the relation between CMV and CABYV is additive, and ii) the relationship between water stress and virus infection is antagonistic, i.e., water stress primes plants for enhanced tolerance to virus infection. Water stress increased leaf greenness and temperature, and reduced leaf water potential, shoot biomass, stem dimensions, rate of flowering, CABYV symptom severity, and marketable fruit yield. Virus infection reduced leaf water potential transiently in single infected plants and persistently until harvest in double-infected plants. Double-virus infection caused the largest and synergistic reduction of marketable fruit yield. The relationship between water regime and virus treatment was additive in 12 out of 15 traits at harvest, with interactions for leaf water content, leaf:stem ratio, and fruit set. We conclude that both virus-virus relations in double infection and virus-drought relations cannot be generalized because they vary with virus, trait, and plant ontogeny.

Exploring the Associations of Afterschool Science Participation and Friendships with Science Identities

Building on previous research that demonstrates the association of youth experiences in afterschool science and higher science identities, this paper presents a network study of 421 middle school students that examines afterschool science participation, friendship ties, and science identities. Participation in afterschool science clubs is associated with higher science identity, but the mechanisms and order of causality are unclear. Youth form friendships inside and outside of school, and peers may influence participation in afterschool activities, as empirical research on friendships shows that they are associated with youth interests. These peer interactions also have the potential to shape identity development during adolescence. In this study, we explore associations among youth participation in afterschool science clubs, peer friendship groups, and science identity. We find that youth who participate in afterschool science clubs have higher science identities than those who do not participate. Additionally, having friends in afterschool science clubs is associated with higher science identity, even among students who report not participating in clubs themselves. Results suggest that afterschool science clubs support youth science identities, even beyond those who directly participate.

The reports of specific waking-state oral behaviours, including awake bruxism activities, and psychological distress have a dose-response relationship: A retrospective medical record study

OBJECTIVE

To investigate if there are dose-response relationships between self-reported waking-state oral behaviours, including awake bruxism, and three indicators of psychological distress (depression, anxiety, stress).

METHODS

The study sample consisted of 1,886 patients with function-dependent TMD pain. Relationships between six non-functional and six functional waking-state oral behaviours, scored on a 5-point ordinal scale, and the psychological factors were investigated using ordinal logistic regression.

RESULTS

Mean age was 42.4 (±15.3) years, 78.7% being female. The odds of reporting the higher categories of non-functional oral behaviours depended on the severity of depression, anxiety, and stress. Most OR coefficients followed a quadratic dose-response distribution, the others increased linearly as the severity of the psychological scales increased. Almost no such associations were found with normal jaw function behaviours.

CONCLUSION

Within the limitations of this study, it may be concluded that non-functional waking-state oral behaviours, including awake bruxism, and psychological distress have a dose-response relationship, with higher levels of distress being associated with higher reports of oral behaviours.

Sex based differences in depression, anxiety, and quality of life and predictors of quality of life among South Asian individuals with chronic obstructive pulmonary disease: A Bayesian analysis

BACKGROUND

COPD can affect both men and women leading to deteriorating impact on general well-being, personal, and family life and decreased quality of life. Anxiety, depression, and self-care behaviours can affect the quality of life of individuals with COPD. There is a dearth of sex-based comparative analyses of anxiety, depression, quality of life, and predictors of quality of life among South Asian individuals with COPD.

PURPOSE

To identify the sex-based differences in depression, anxiety, and quality of life and the predictors of quality of life among South Asian individuals with COPD.

METHODS

A cross-sectional survey of 294 men and 114 women with COPD was conducted in Khyber Pakhtunkhwa, Pakistan. Hospital Anxiety and Depression scale, World Health Organization Quality of Life-brief version, Self-Care of Chronic Obstructive Pulmonary Disease Inventory, the Self-Care Self-Efficacy in COPD Scale were used for data collection. Bayesian independent sample t-test was used to compare mean differences in depression, anxiety, and quality of life among men and women. Two regression models were examined to determine if age, years of living with COPD, anxiety, depression, self-care self-efficacy, self-care monitoring, management, and maintenance were predictors of quality of life among men and women.

FINDINGS

Bayesian analysis showed anecdotal evidence that women had higher levels of depression, but lower levels of anxiety compared to men. Anecdotal evidence indicated that the physical quality of life of men was better than women, but strong evidence that their social relationship and environmental quality of life was better than women. Years of living with COPD, anxiety, depression, self-care self-efficacy, self-care management, self-care monitoring, and self-care maintenance were stronger predictors of women's quality of life. Anxiety and depression only predicted men's psychological quality of life, but predicted women's psychological, social relationships, and environmental quality of life.

CONCLUSIONS

The findings contribute to literature highlighting sex-based differences in anxiety, depression, and quality of life among South Asian men and women with COPD. Men generally reported higher levels of quality of life than women across all domains. Women's social relationships and environmental quality of life were greatly impacted by anxiety and depression. Quality of life interventions for women should be targeted at improving their social relationships and environmental satisfaction and addressing anxiety and depression.

Impact of redefining statistical significance on P-hacking and false positive rates: An agent-based model

In recent years, concern has grown about the inappropriate application and interpretation of P values, especially the use of P<0.05 to denote "statistical significance" and the practice of P-hacking to produce results below this threshold and selectively reporting these in publications. Such behavior is said to be a major contributor to the large number of false and non-reproducible discoveries found in academic journals. In response, it has been proposed that the threshold for statistical significance be changed from 0.05 to 0.005. The aim of the current study was to use an evolutionary agent-based model comprised of researchers who test hypotheses and strive to increase their publication rates in order to explore the impact of a 0.005 P value threshold on P-hacking and published false positive rates. Three scenarios were examined, one in which researchers tested a single hypothesis, one in which they tested multiple hypotheses using a P<0.05 threshold, and one in which they tested multiple hypotheses using a P<0.005 threshold. Effects sizes were varied across models and output assessed in terms of researcher effort, number of hypotheses tested and number of publications, and the published false positive rate. The results supported the view that a more stringent P value threshold can serve to reduce the rate of published false positive results. Researchers still engaged in P-hacking with the new threshold, but the effort they expended increased substantially and their overall productivity was reduced, resulting in a decline in the published false positive rate. Compared to other proposed interventions to improve the academic publishing system, changing the P value threshold has the advantage of being relatively easy to implement and could be monitored and enforced with minimal effort by journal editors and peer reviewers.

Accuracy of Intraset Repetitions-in-Reserve Predictions During the Bench Press Exercise in Resistance-Trained Male and Female Subjects

ABSTRACT

Refalo, MC, Remmert, JF, Pelland, JC, Robinson, ZP, Zourdos, MC, Hamilton, DL, Fyfe, JJ, and Helms, ER. Accuracy of intraset repetitions-in-reserve predictions during the bench press exercise in resistance-trained male and female subjects. J Strength Cond Res 38(3): e78-e85, 2024-This study assessed the accuracy of intraset repetitions-in-reserve (RIR) predictions to provide evidence for the efficacy of RIR prescription as a set termination method to inform proximity to failure during resistance training (RT). Twenty-four resistance trained male ( n = 12) and female ( n = 12) subjects completed 2 experimental sessions involving 2 sets performed to momentary muscular failure (barbell bench press exercise) with 75% of 1 repetition maximum (1RM), whereby subjects verbally indicated when they perceived to had reached either 1 RIR or 3 RIR. The difference between the predicted RIR and the actual RIR was defined as the "RIR accuracy" and was quantified as both raw (i.e., direction of error) and absolute (i.e., magnitude of error) values. High raw and absolute mean RIR accuracy (-0.17 ± 1.00 and 0.65 ± 0.78 repetitions, respectively) for 1-RIR and 3-RIR predictions were observed (including all sets and sessions completed). We identified statistical equivalence (equivalence range of ±1 repetition, thus no level of statistical significance was set) in raw and absolute RIR accuracy between (a) 1-RIR and 3-RIR predictions, (b) set 1 and set 2, and (c) session 1 and session 2. No evidence of a relationship was found between RIR accuracy and biological sex, years of RT experience, or relative bench press strength. Overall, resistance-trained individuals are capable of high absolute RIR accuracy when predicting 1 and 3 RIR on the barbell bench press exercise, with a minor tendency for underprediction. Thus, RIR prescriptions may be used in research and practice to inform the proximity to failure achieved upon set termination.

Care pathways of sepsis survivors: sequelae, mortality and use of healthcare services in France, 2015-2018

BACKGROUND

Individuals who survive sepsis are at high risk of chronic sequelae, resulting in significant health-economic costs. Several studies have focused on aspects of healthcare pathways of sepsis survivors but comprehensive, longitudinal overview of their pathways of care are scarce. The aim of this retrospective, longitudinal cohort study is to identify sepsis survivor profiles based on their healthcare pathways and describe their healthcare consumption and costs over the 3 years following their index hospitalization.

METHODS

The data were extracted from the French National Hospital Discharge Database. The study population included all patients above 15 years old, with bacterial sepsis, who survived an incident hospitalization in an acute care facility in 2015. To identify survivor profiles, state sequence and clustering analyses were conducted over the year following the index hospitalization. For each profile, patient characteristics and their index hospital stay and sequelae were described, as well as use of care and its associated monetary costs, both pre- and post-sepsis.

RESULTS

New medical (79.2%), psychological (26.9%) and cognitive (18.5%) impairments were identified post-sepsis, and 65.3% of survivors were rehospitalized in acute care. Cumulative mortality reached 36.6% by 3 years post-sepsis. The total medical cost increased by 856 million € in the year post-sepsis. Five patient clusters were identified: home (65.6% of patients), early death (12.9%), late death (6.8%), short-term rehabilitation (11.3%) and long-term rehabilitation (3.3%). Survivors with early and late death clusters had high rates of cancer and primary bacteremia and experienced more hospital-at-home care post-sepsis. Survivors in short- or long-term rehabilitation clusters were older, with higher percentage of septic shock than those coming back home, and had high rates of multiple site infections and higher rates of new psychological and cognitive impairment.

CONCLUSIONS

Over three years post-sepsis, different profiles of sepsis survivors were identified with different mortality rates, sequels and healthcare services usage and cost. This study confirmed the importance of sepsis burden and suggests that strategies of post-discharge care, in accordance with patient profile, should be further tested in order to reduce sepsis burden.

Trials and tribulations of statistical significance in biochemistry and omics

Over recent years many statisticians and researchers have highlighted that statistical inference would benefit from a better use and understanding of hypothesis testing, p-values, and statistical significance. We highlight three recommendations in the context of biochemical sciences. First recommendation: to improve the biological interpretation of biochemical data, do not use p-values (or similar test statistics) as thresholded values to select biomolecules. Second recommendation: to improve comparison among studies and to achieve robust knowledge, perform complete reporting of data. Third recommendation: statistical analyses should be reported completely with exact numbers (not as asterisks or inequalities). Owing to the high number of variables, a better use of statistics is of special importance in omic studies.

Effect sizes and effect size benchmarks in family violence research

Scholarly journals increasingly request that authors include effect size (ES) estimates when reporting statistical results. However, there is little guidance on how authors should interpret ESs. Consequently, some authors do not provide ES interpretations, or, when interpretations are provided, they often fail to use appropriate reference groups, using instead the ES benchmarks suggested by Cohen (1988). After discussing the most commonly used ES estimates, we describe the method used by Cohen (1962) to develop ES benchmarks (i.e., small, medium, and large) for use in power analyses and describe the limitations associated with using these benchmarks. Next, we establish general benchmarks for family violence (FV) research. That is, we followed Cohen's approach to establishing his original ES benchmarks using family violence research published in 2021 in Child Abuse & Neglect, which produced a medium ES (d = 0.354) that was smaller than Cohen's recommended medium ES (d = 0.500). Then, we examined the ESs in different subspecialty areas of FV research to provide benchmarks for contextualized FV ESs and to provide information that can be used to conduct power analyses when planning future FV research. Finally, some of the challenges to developing ES benchmarks in any scholarly discipline are discussed. For professionals who are not well informed about ESs, the present review is designed to increase their understanding of ESs and what ES benchmarks tell them (and do not tell them) with respect to understanding the meaningfulness of FV research findings.

Influence of surrogate scalp material and thickness on head impact responses: Toward a biofidelic head-brain physical model

Advanced physical head models capable of replicating both global kinematics and intracranial mechanics of the human head are required for head injury research and safety gear assessment. These head surrogates require a complex design to accommodate realistic anatomical details. The scalp is a crucial head component, but its influence on the biomechanical response of such head surrogates remains unclear. This study aimed to evaluate the influence of surrogate scalp material and thickness on head accelerations and intraparenchymal pressures using an advanced physical head-brain model. Scalp pads made from four materials (Vytaflex20, Vytaflex40, Vytaflex50, PMC746) and each material with four thicknesses (2, 4, 6, and 8 mm) were evaluated. The head model attached to the scalp pad was dropped onto a rigid plate from two heights (5 and 19.5 cm) and at three head locations (front, right side, and back). While the selected materials' modulus exhibited a relatively minor effect on head accelerations and coup pressures, the effect of scalp thickness was shown to be major. Moreover, by decreasing the thickness of the head's original scalp by 2 mm and changing the original scalp material from Vytaflex 20 to Vytaflex 40 or Vytaflex 50, the head acceleration biofidelity ratings could improve by 30% and approached the considered rating (0.7) of good biofidelity. This study provides a potential direction for improving the biofidelity of a novel head model that might be a useful tool in head injury research and safety gear tests. This study also has implications for selecting appropriate surrogate scalps in the future design of physical and numerical head models.

Bayesian hypothesis testing of mediation: Methods and the impact of prior odds specifications

Mediation analysis is widely used to study whether the effect of an independent variable on an outcome is transmitted through a mediator. Bayesian methods have become increasingly popular for mediation analysis. However, limited research has been done on formal Bayesian hypothesis testing of mediation. Although hypothesis testing using Bayes factor for a single path is readily available, how to integrate the Bayes factors of two paths (from input to mediator and from mediator to outcome) while incorporating prior beliefs on the two paths and/or mediation is under-studied. In the current study, we propose a general approach to Bayesian hypothesis testing of mediation. The proposed approach allows researchers to specify prior odds based on the substantive research context and can be used in mediation modeling with latent variables. The impact of prior odds specifications on Bayesian hypothesis test of mediation is demonstrated via both real and hypothetical data examples. Both R functions and a user-friendly R web app are provided for the implementation of the proposed approach. Our study can add to researchers' toolbox of mediation analysis and raise researchers' awareness of the importance of prior odds specifications in Bayesian hypothesis testing of mediation.

Let's Talk about Stats: Revising Our Approach to Teaching Statistics in Psychology

Undergraduate statistics in psychology is an important, often challenging, course for students. The focus in psychology tends to be on hypothesis tests, such as t tests and analysis of variance. While adequate for some questions, there are many other topics we might include that could improve that data analytic abilities of students and improve psychological science in the long run. Topics such as generalized linear modeling, multilevel modeling, Bayesian statistics, model building and comparison, and causal analysis, could be introduced in an undergraduate psychological statistics course. For each topic, I discuss their importance and provide sources for instructor's continuing education. These topics would give students greater flexibility in analyzing data, allow them to conduct more meaningful analyses, allow them to understand more modern data analytic approaches, and potentially help the field of psychology in the long run, by being one part of the strategy to address the reproducibility problem.

Chloride-dependent mechanisms of multimodal sensory discrimination and nociceptive sensitization in Drosophila

Individual sensory neurons can be tuned to many stimuli, each driving unique, stimulus-relevant behaviors, and the ability of multimodal nociceptor neurons to discriminate between potentially harmful and innocuous stimuli is broadly important for organismal survival. Moreover, disruptions in the capacity to differentiate between noxious and innocuous stimuli can result in neuropathic pain. Drosophila larval class III (CIII) neurons are peripheral noxious cold nociceptors and innocuous touch mechanosensors; high levels of activation drive cold-evoked contraction (CT) behavior, while low levels of activation result in a suite of touch-associated behaviors. However, it is unknown what molecular factors underlie CIII multimodality. Here, we show that the TMEM16/anoctamins subdued and white walker (wwk; CG15270) are required for cold-evoked CT, but not for touch-associated behavior, indicating a conserved role for anoctamins in nociception. We also evidence that CIII neurons make use of atypical depolarizing chloride currents to encode cold, and that overexpression of ncc69-a fly homologue of NKCC1-results in phenotypes consistent with neuropathic sensitization, including behavioral sensitization and neuronal hyperexcitability, making Drosophila CIII neurons a candidate system for future studies of the basic mechanisms underlying neuropathic pain.

Replication concerns in sports and exercise science: a narrative review of selected methodological issues in the field

Known methodological issues such as publication bias, questionable research practices and studies with underpowered designs are known to decrease the replicability of study findings. The presence of such issues has been widely established across different research fields, especially in psychology. Their presence raised the first concerns that the replicability of study findings could be low and led researchers to conduct large replication projects. These replication projects revealed that a significant portion of original study findings could not be replicated, giving rise to the conceptualization of the replication crisis. Although previous research in the field of sports and exercise science has identified the first warning signs, such as an overwhelming proportion of significant findings, small sample sizes and lack of data availability, their possible consequences for the replicability of our field have been overlooked. We discuss the consequences of the above issues on the replicability of our field and offer potential solutions to improve replicability.

One-year hospital readmission for recurrent sepsis: associated risk factors and impact on 1-year mortality-a French nationwide study

BACKGROUND

Sepsis is a complex health condition, leading to long-term morbidity and mortality. Understanding the risk factors for recurrent sepsis, as well as its impact on mid- and long-term mortality among other risk factors, is essential to improve patient survival.

METHODS

A risk factor analysis, based on French nationwide medico-administrative data, was conducted on a cohort of patients above 15 years old, hospitalized with an incident sepsis in metropolitan France between 1st January 2018 and 31st December 2018 and who survived their index hospitalization. Two main analyses, focusing on outcomes occurring 1-year post-discharge, were conducted: a first one to assess risk factors for recurrent sepsis and a second to assess risk factors for mortality.

RESULTS

Of the 178017 patients surviving an incident sepsis episode in 2018 and included in this study, 22.3% died during the 1-year period from discharge and 73.8% had at least one hospital readmission in acute care, among which 18.1% were associated with recurrent sepsis. Patients aged between 56 and 75, patients with cancer and renal disease, with a long index hospital stay or with mediastinal or cardiac infection had the highest odds of recurrent sepsis. One-year mortality was higher for patients with hospital readmission for recurrent sepsis (aOR 2.93; 99% CI 2.78-3.09). Among all comorbidities, patients with cancer (aOR 4.35; 99% CI 4.19-4.52) and dementia (aOR 2.02; 99% CI 1.90-2.15) had the highest odds of 1-year mortality.

CONCLUSION

Hospital readmission for recurrent sepsis is one of the most important risk factors for 1-year mortality of septic patients, along with age and comorbidities. Our study suggests that recurrent sepsis, as well as modifiable or non-modifiable other risk factors identified, should be considered in order to improve patient care pathway and survival.

Digital Devices for Assessing Motor Functions in Mobility-Impaired and Healthy Populations: Systematic Literature Review

Background

With the advent of smart sensing technology, mobile and wearable devices can provide continuous and objective monitoring and assessment of motor function outcomes.

Objective

We aimed to describe the existing scientific literature on wearable and mobile technologies that are being used or tested for assessing motor functions in mobility-impaired and healthy adults and to evaluate the degree to which these devices provide clinically valid measures of motor function in these populations.

Methods

A systematic literature review was conducted by searching Embase, MEDLINE, CENTRAL (January 1, 2015, to June 24, 2020), the United States and European Union clinical trial registries, and the United States Food and Drug Administration website using predefined study selection criteria. Study selection, data extraction, and quality assessment were performed by 2 independent reviewers.

Results

A total of 91 publications representing 87 unique studies were included. The most represented clinical conditions were Parkinson disease (n=51 studies), followed by stroke (n=5), Huntington disease (n=5), and multiple sclerosis (n=2). A total of 42 motion-detecting devices were identified, and the majority (n=27, 64%) were created for the purpose of health care–related data collection, although approximately 25% were personal electronic devices (eg, smartphones and watches) and 11% were entertainment consoles (eg, Microsoft Kinect or Xbox and Nintendo Wii). The primary motion outcomes were related to gait (n=30), gross motor movements (n=25), and fine motor movements (n=23). As a group, sensor-derived motion data showed a mean sensitivity of 0.83 (SD 7.27), a mean specificity of 0.84 (SD 15.40), a mean accuracy of 0.90 (SD 5.87) in discriminating between diseased individuals and healthy controls, and a mean Pearson r validity coefficient of 0.52 (SD 0.22) relative to clinical measures. We did not find significant differences in the degree of validity between in-laboratory and at-home sensor-based assessments nor between device class (ie, health care–related device, personal electronic devices, and entertainment consoles).

Conclusions

Sensor-derived motion data can be leveraged to classify and quantify disease status for a variety of neurological conditions. However, most of the recent research on digital clinical measures is derived from proof-of-concept studies with considerable variation in methodological approaches, and much of the reviewed literature has focused on clinical validation, with less than one-quarter of the studies performing analytical validation. Overall, future research is crucially needed to further consolidate that sensor-derived motion data may lead to the development of robust and transformative digital measurements intended to predict, diagnose, and quantify neurological disease state and its longitudinal change.

Should anthropometric differences between the commonly used pedestrian computational biomechanics models and Chinese population be taken into account when predicting pedestrian head kinematics and injury in vehicle collisions in China?

Computational biomechanics models play a key role in predicting/evaluating pedestrian head kinematics and injury risk in car-to-pedestrian collisions. The human multibody models most commonly used in car-to-pedestrian collision reconstruction, such as pedestrian model by The Netherlands Organisation for Applied Scientific Research TNO, are built using the anthropometry of Western European population as defined in TNO (2013) human multibody model database. In this study, we investigate the effects of the anthropometric differences between the Western European and Chinese populations on the pedestrian head kinematics and injury responses predicted using multibody models. The comparison was conducted through car-to-pedestrian collision simulations using pedestrian multibody models representing anthropometric characteristics of Western European and Chinese populations, three typical vehicle shapes (sedan, SUV and minivan), five initial vehicle impact speeds (30, 35, 40, 45, 50 km/h), and six pedestrian walking postures. The results indicate that the change of pedestrian model anthropometry (from Western European to Chinese) exerts appreciable effects on both the predicted initial boundary conditions of the head-to-windscreen impact (in particular the head-to-windscreen impact angle) and the head injury indices in the impact with the road surface (secondary impact).

The 40-Hz auditory steady-state response in bipolar disorder: A meta-analysis

OBJECTIVE

Bipolar disorder is characterized by aberrant neurophysiological responses as measured with electroencephalography (EEG) and magnetoencephalography (MEG), including the 40-Hz auditory steady-state response (ASSR). 40-Hz ASSR deficits are also found in patients with schizophrenia and may represent a transdiagnostic biomarker of neuronal circuit dysfunction. In this systematic review and meta-analysis, we summarize and evaluate the evidence for 40-Hz ASSR deficits in patients with bipolar disorder.

METHODS

We identified studies from PubMed, EMBASE, and SCOPUS. We assessed the risk of bias, calculated Hedges' g meta-level effect sizes, and investigated small-study effects using funnel plots and Egger regression.

RESULTS

Seven studies, comprising 396 patients with bipolar disorder and 404 healthy controls, were included in the meta-analysis. Studies displayed methodological heterogeneity and an overall high risk of bias. Patients with bipolar disorder showed consistent reductions in 40-Hz ASSR evoked power (Hedges' g = -0.49; 95% confidence intervals [-0.67, -0.31]) and inter-trial phase coherence (ITPC) (Hedges' g = -0.43; 95 %CI [-0.58, -0.29]) compared with healthy controls.

CONCLUSIONS

Our meta-analysis provides evidence that 40-Hz ASSRs are reduced in patients with bipolar disorder compared with healthy controls.

SIGNIFICANCE

Future large-scale studies are warranted to link 40-Hz ASSR deficits to clinical features and developmental trajectories.

Mechanistic Research for the Student or Educator (Part I of II)

Many discoveries in the biological sciences have emerged from observational studies, but student researchers also need to learn how to design experiments that distinguish correlation from causation. For example, identifying the physiological mechanism of action of drugs with therapeutic potential requires the establishment of causal links. Only by specifically interfering with the purported mechanisms of action of a drug can the researcher determine how the drug causes its physiological effects. Typically, pharmacological or genetic approaches are employed to modify the expression and/or activity of the biological drug target or downstream pathways, to test if the salutary properties of the drug are thereby abolished. However, experimental techniques have caveats that tend to be underappreciated, particularly for newer methods. Furthermore, statistical effects are no guarantor of their biological importance or translatability across models and species. In this two-part series, the caveats and strengths of mechanistic preclinical research are briefly described, using the intuitive example of pharmaceutical drug testing in experimental models of human diseases. Part I focuses on technical practicalities and common pitfalls of cellular and animal models designed for drug testing, and Part II describes in simple terms how to leverage a full-factorial ANOVA, to test for causality in the link between drug-induced activation (or inhibition) of a biological target and therapeutic outcomes. Upon completion of this series, students will have forehand knowledge of technical and theoretical caveats in mechanistic research, and comprehend that “a model is just a model.” These insights can help the new student appreciate the strengths and limitations of scientific research.

Helping reviewers assess statistical analysis: A case study from analytic methods

Analytic methods development, like many other disciplines, relies on experimentation and data analysis. Determining the contribution of a paper or report on a study incorporating data analysis is typically left to the reviewer's experience and good sense, without reliance on structured guidelines. This is amplified by the growing role of machine learning driven analysis, where results are based on computer intensive algorithm applications. The evaluation of a predictive model where cross validation was used to fit its parameters adds challenges to the evaluation of regression models, where the estimates can be easily reproduced. This lack of structure to support reviews increases uncertainty and variability in reviews. In this paper, aspects of statistical assessment are considered. We provide checklists for reviewers of applied statistics work with a focus on analytic method development. The checklist covers six aspects relevant to a review of statistical analysis, namely: (1) study design, (2) algorithmic and inferential methods in frequentism analysis, (3) Bayesian methods in Bayesian analysis (if relevant), (4) selective inference aspects, (5) severe testing properties and (6) presentation of findings. We provide a brief overview of these elements providing references for a more elaborate treatment. The robustness analysis of an analytical method is used to illustrate how an improvement can be achieved in response to questions in the checklist. The paper is aimed at both engineers and seasoned researchers.

Testing the Magnitude of Correlations Across Experimental Conditions

Correlation coefficients are often compared to investigate data across multiple research fields, as they allow investigators to determine different degrees of correlation to independent variables. Even with adequate sample size, such differences may be minor but still scientifically relevant. To date, although much effort has gone into developing methods for estimating differences across correlation coefficients, adequate tools for variable sample sizes and correlational strengths have yet to be tested. The present study evaluated four different methods for detecting the difference between two correlations and tested the adequacy of each method using simulations with multiple data structures. The methods tested were Cohen's q, Fisher's method, linear mixed-effects models (LMEM), and an ad hoc developed procedure that integrates bootstrap and effect size estimation. Correlation strengths and sample size was varied across a wide range of simulations to test the power of the methods to reject the null hypothesis (i.e., the two correlations are equal). Results showed that Fisher's method and the LMEM failed to reject the null hypothesis even in the presence of relevant differences between correlations and that Cohen's method was not sensitive to the data structure. Bootstrap followed by effect size estimation resulted in a fair, unbiased compromise for estimating quantitative differences between statistical associations and producing outputs that could be easily compared across studies. This unbiased method is easily implementable in MatLab through the bootes function, which was made available online by the author at MathWorks.

Trends in bacterial sepsis incidence and mortality in France between 2015 and 2019 based on National Health Data System (Système National des données de Santé (SNDS)): a retrospective observational study

OBJECTIVE

This study aims to provide a case definition of sepsis of presumed bacterial aetiology based on 10th revision of the International Classification of Diseases (ICD-10) codes, to assess trends in sepsis incidence and mortality between 2015 and 2019 in France, and to describe the characteristics of affected patients and hospital stays.

DESIGN

Nationwide, population-based, retrospective observational study.

SETTING

Metropolitan France between 2015 and 2019.

PARTICIPANTS

Between 2015 and 2019, 1 224 433 patients with sepsis of presumed bacterial aetiology were selected from the French National Hospital Discharge Database (Programme de Médicalisation des Systèmes d'Information) and were identified from corresponding ICD-10 codes for explicit sepsis or implicit sepsis.

MAIN OUTCOMES MEASURES

Annual overall and age-specific and gender-specific incidence and 95% CI, as well as trends in sepsis incidence and mortality, were estimated. Comorbidities, length of hospital stay and outcomes were described.

RESULTS

The sex-standardised and age-standardised incidence per 100 000 (95% CI) increased from 357 (356.0 to 359.0) in 2015 to 403 (401.9 to 405.0) in 2019 and remained higher for males compared with females. Children under 1 year and patients over 75 years consistently had the highest incidence. The most common comorbidities were cancer and chronic heart failure. The median hospital length of stay was 12 days. Most patients came from home, but only half returned home after their hospital stay and approximately 15% were discharged to long-term care. In-hospital mortality was about 25% and declined along the study period.

CONCLUSIONS

Medico-administrative databases can be used to provide nationwide estimates of the in-hospital burden of bacterial sepsis. The results confirm the high burden of sepsis in France. These data should be complemented by estimating the additional burden associated with fungal and viral infections during the COVID-19 pandemic.

Reproducibility, relevance and reliability as barriers to efficient and credible biomedical technology translation

Biomedical research accuracy and relevance for improving healthcare are increasingly identified as costly problems. Basic research data quality, reporting and methodology, and reproducibility are common factors implicated in this challenge. Preclinical models of disease and therapy, largely conducted in rodents, have known deficiencies in replicating most human conditions. Their translation to human results is acknowledged to be poor for decades. Clinical data quality and quantity is also recognized as deficient; gold standard randomized clinical trials are expensive. Few solid conclusions from clinical studies are replicable and many remain unpublished. The translational pathway from fundamental biomedical research through to innovative solutions handed to clinical practitioners is therefore highly inefficient and costly in terms of wasted resources, early claims from fundamental discoveries never witnessed in humans, and few new, improved solutions available clinically for myriad diseases. Improving this biomedical research strategy and resourcing for reliability, translational relevance, reproducibility and clinical impact requires careful analysis and consistent enforcement at both funding and peer review levels.

Critical Care Randomized Trials Demonstrate Power Failure: A Low Positive Predictive Value of Findings in the Critical Care Research Field

BACKGROUND

We aimed to determine the post-hoc power of randomized controlled trials (RCTs) in critical care, and describe the implications for long-term positive (PPV) and negative predictive value (NPV) of statistically significant and non-significant findings respectively in the research field.

METHODS

We reviewed three cohorts of RCTs. "Adult-RCTs" were 216 multicenter RCTs with a mortality outcome from a published systematic review. "Pediatric-RCTs" were 120 RCTs with a mortality outcome, obtained by search of picutrials.net. "Consecutive-RCTs" were 90 recent RCTs obtained by screening publications in 6 journals. Post-hoc power for each study was calculated at α 0.05 and 0.005, for measures of small, medium, and large effect-size, using G*Power software. Long-run expected PPV and NPV of critical care research field findings were then calculated.

RESULTS

With α 0.05, post-hoc power for small effect-size was very low in all RCT-cohorts (eg, median 24% in Adult-RCTs). For medium effect-size, post-hoc power was low, except for Adult-RCTs (eg, median 9% in Pediatric-RCTs). For large effect-size, post-hoc power for non-human-animal Consecutive-RCTs was low (median 32%). With α 0.005, post-hoc power was even lower. The corollary was that both PPV and NPV were poor for small effect-size, unless α 0.005 was used. Even with α 0.005, with realistic (vs. optimistic) prior probability of the alternative hypothesis, the PPV was low (eg, in Adult-RCTs 57.1% vs. 92.3%). Adding mild bias (0.1) reduced the PPV even further. For medium effect-size both PPV and NPV were better; nevertheless, with α 0.05 and realistic prior probability of the alternative hypothesis the PPV was poor, and with α 0.005 and mild bias (0.1) the PPV was very low (eg, Adult-RCTs median 44.1%).

CONCLUSIONS

To improve the predictive value of findings in the critical care research field, RCTs should be designed to have 80% power for realistic effect-size at α 0.005.

Accumulating Knowledge in the Organizational Sciences

In some fields, research findings are rigorously curated in a common language and made available to enable future use and large-scale, robust insights. Organizational researchers have begun such efforts [e.g., metaBUS ( http://metabus.org/ )] but are far from the efficient, comprehensive curation seen in areas such as cognitive neuroscience or genetics. This review provides a sample of insights from research curation efforts in organizational research, psychology, and beyond—insights not possible by even large-scale, substantive meta-analyses. Efforts are classified as either science-of-science research or large-scale, substantive research. The various methods used for information extraction (e.g., from PDF files) and classification (e.g., using consensus ontologies) is reviewed. The review concludes with a series of recommendations for developing and leveraging the available corpus of organizational research to speed scientific progress.

Providing Evidence for the Null Hypothesis in Functional Magnetic Resonance Imaging Using Group-Level Bayesian Inference

Classical null hypothesis significance testing is limited to the rejection of the point-null hypothesis; it does not allow the interpretation of non-significant results. This leads to a bias against the null hypothesis. Herein, we discuss statistical approaches to ‘null effect’ assessment focusing on the Bayesian parameter inference (BPI). Although Bayesian methods have been theoretically elaborated and implemented in common neuroimaging software packages, they are not widely used for ‘null effect’ assessment. BPI considers the posterior probability of finding the effect within or outside the region of practical equivalence to the null value. It can be used to find both ‘activated/deactivated’ and ‘not activated’ voxels or to indicate that the obtained data are not sufficient using a single decision rule. It also allows to evaluate the data as the sample size increases and decide to stop the experiment if the obtained data are sufficient to make a confident inference. To demonstrate the advantages of using BPI for fMRI data group analysis, we compare it with classical null hypothesis significance testing on empirical data. We also use simulated data to show how BPI performs under different effect sizes, noise levels, noise distributions and sample sizes. Finally, we consider the problem of defining the region of practical equivalence for BPI and discuss possible applications of BPI in fMRI studies. To facilitate ‘null effect’ assessment for fMRI practitioners, we provide Statistical Parametric Mapping 12 based toolbox for Bayesian inference.

Misleading Reporting (Spin) in Noninferiority Randomized Clinical Trials in Oncology With Statistically Not Significant Results: A Systematic Review

IMPORTANCE

Spin, the inaccurate reporting of randomized clinical trials (RCTs) with results that are not statistically significant for the primary end point, distorts interpretation of results and leads to misinterpretation. However, the prevalence of spin and related factors in noninferiority cancer RCTs remains unclear.

OBJECTIVE

To examine misleading reporting, or spin, and the associated factors in noninferiority cancer RCTs through a systematic review.

DATA SOURCES

A systematic search of the PubMed database was performed for articles published between January 1, 2010, and December 31, 2019, using the Cochrane Highly Sensitive Search Strategy.

STUDY SELECTION

Two investigators independently selected studies using the inclusion criteria of noninferiority parallel-group RCTs aiming to confirm effects to cancer treatments published between January 1, 2010, and December 31, 2019, reporting results that were not statistically significant for the primary end points.

DATA EXTRACTION AND SYNTHESIS

Standardized data abstraction was used to extract information concerning the trial characteristics and spin based on a prespecified definition. The main investigator extracted the trial characteristics while both readers independently evaluated the spin. The Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) reporting guideline was followed.

MAIN OUTCOMES AND MEASURES

The main outcome was spin prevalence in any section of the report. Spin was defined as use of specific reporting strategies, from whatever motive, to highlight that the experimental treatment is beneficial, despite no statistically significant difference for the primary outcome, or to distract the reader from results that are not statistically significant. The associations (prevalence difference and odds ratios [ORs]) between spin and trial characteristics were also evaluated.

RESULTS

The analysis included 52 of 2752 reports identified in the PubMed search. Spin was identified in 39 reports (75.0%; 95% CI, 61.6%-84.9%), including the abstract (34 reports [65.4%; 95% CI, 51.1%-76.9%]) and the main text (38 reports [73.1%; 95% CI, 59.7%-83.3%]). Univariate analysis found that the spin prevalence was higher in reports with data managers (prevalence difference, 27%; 95% CI, 1.1%-50.3%), reports without funding from for-profit sources (prevalence difference, 31.2%; 95% CI, 4.8%-53.8%), and reports of novel experimental treatments (prevalence difference, 37.5%; 95% CI, 5.8%-64.7%). Multivariable analysis found that novel experimental treatment (OR, 4.64; 95% CI, 0.98-22.02) and funding only from nonprofit sources only (OR, 5.20; 95% CI, 1.21-22.29) were associated with spin.

CONCLUSIONS AND RELEVANCE

In this systematic review, most noninferiority RCTs reporting results that were not statistically significant for the primary end points showed distorted interpretation and inaccurate reporting. The novelty of an experimental treatment and funding only from nonprofit sources were associated with spin.

[Clinical studies in trauma surgery and orthopedics: read, interpret and implement]

Für eine informierte, partizipative ärztliche Entscheidungsfindung müssen Sie neben Ihrem Erfahrungswissen aktuelle wissenschaftliche Daten berücksichtigen. Die digitale Informationsflut macht es im klinischen Alltag allerdings schwer, immer auf dem Stand des Wissens zu bleiben. Der Beitrag bietet einige Hilfen an, wie mit diesem Problem umgegangen werden kann. Ein grundlegendes Verständnis über Vor- und Nachwahrscheinlichkeiten sowie systematische Fehler (Bias) erleichtert die Abwägung zwischen Nutzen und Risiko z. B. einer (chirurgischen) Intervention im Vergleich zu einer nichtoperativen Therapie. Randomisierte Studien („randomized controlled trials“, RCT, mit allen modernen Modifikationen) liefern unverzerrte Ergebnisse, führen in Orthopädie und Unfallchirurgie evtl. zu einer stark selektierten, nichtrepräsentativen Stichprobe, und ihre Resultate müssen durch weitere, unabhängige RCT bestätigt oder widerlegt werden. Große beobachtende Datenmengen (z. B. aus Registern) können quasiexperimentell modelliert werden und RCT in der Gesundheitstechnologiebewertung flankieren.

Reprocessing 16S rRNA Gene Amplicon Sequencing Studies: (Meta)Data Issues, Robustness, and Reproducibility

High-throughput sequencing technology provides an efficient method for evaluating microbial ecology. Different bioinformatics pipelines can be used to convert 16S ribosomal RNA gene amplicon sequencing data into an operational taxonomic unit (OTU) table that is used to analyze microbial communities. It is important to assess the robustness of these pipelines, each with specific algorithms and/or parameters, and their influence on the outcome of statistical tests. Articles with publicly available datasets on the oral microbiome were searched for, and five datasets were retrieved. These were from studies on changes in microbiota related to smoking, oral cancer, caries, diabetes, or periodontitis. Next, the data was processed with four pipelines based on VSEARCH, USEARCH, mothur, and UNOISE3. OTU tables were rarefied, and differences in α-diversity and β-diversity were tested for different groups in a dataset. Finally, these results were checked for consistency among these example pipelines. Of articles that deposited data, only 57% made all sequencing and metadata available. When processing the datasets, issues were encountered, caused by read characteristics and differences between tools and their defaults in combination with a lack of detail in the methodology of the articles. In general, the four mainstream pipelines provided similar results, but importantly, P-values sometimes differed between pipelines beyond the significance threshold. Our results indicated that for published articles, the description of bioinformatics methods and data deposition should be improved, and regarding reproducibility, that analysis of multiple subsamples is required when using rarefying as library-size normalization method.

Bayesian Hodges-Lehmann tests for statistical equivalence in the two-sample setting: Power analysis, type I error rates and equivalence boundary selection in biomedical research

BACKGROUND

Null hypothesis significance testing (NHST) is among the most frequently employed methods in the biomedical sciences. However, the problems of NHST and p-values have been discussed widely and various Bayesian alternatives have been proposed. Some proposals focus on equivalence testing, which aims at testing an interval hypothesis instead of a precise hypothesis. An interval hypothesis includes a small range of parameter values instead of a single null value and the idea goes back to Hodges and Lehmann. As researchers can always expect to observe some (although often negligibly small) effect size, interval hypotheses are more realistic for biomedical research. However, the selection of an equivalence region (the interval boundaries) often seems arbitrary and several Bayesian approaches to equivalence testing coexist.

METHODS

A new proposal is made how to determine the equivalence region for Bayesian equivalence tests based on objective criteria like type I error rate and power. Existing approaches to Bayesian equivalence testing in the two-sample setting are discussed with a focus on the Bayes factor and the region of practical equivalence (ROPE). A simulation study derives the necessary results to make use of the new method in the two-sample setting, which is among the most frequently carried out procedures in biomedical research.

RESULTS

Bayesian Hodges-Lehmann tests for statistical equivalence differ in their sensitivity to the prior modeling, power, and the associated type I error rates. The relationship between type I error rates, power and sample sizes for existing Bayesian equivalence tests is identified in the two-sample setting. Results allow to determine the equivalence region based on the new method by incorporating such objective criteria. Importantly, results show that not only can prior selection influence the type I error rate and power, but the relationship is even reverse for the Bayes factor and ROPE based equivalence tests.

CONCLUSION

Based on the results, researchers can select between the existing Bayesian Hodges-Lehmann tests for statistical equivalence and determine the equivalence region based on objective criteria, thus improving the reproducibility of biomedical research.

A permutation-based approach for heterogeneous meta-analyses of rare events

The increasingly widespread use of meta-analysis has led to growing interest in meta-analytic methods for rare events and sparse data. Conventional approaches tend to perform very poorly in such settings. Recent work in this area has provided options for sparse data, but these are still often hampered when heterogeneity across the available studies differs based on treatment group. We propose a permutation-based approach based on conditional logistic regression that accommodates this common contingency, providing more reliable statistical tests when such patterns of heterogeneity are observed. We find that commonly used methods can yield highly inflated Type I error rates, low confidence interval coverage, and bias when events are rare and non-negligible heterogeneity is present. Our method often produces much lower Type I error rates and higher confidence interval coverage than traditional methods in these circumstances. We illustrate the utility of our method by comparing it to several other methods via a simulation study and analyzing an example data set, which assess the use of antibiotics to prevent acute rheumatic fever.

Identification of a neural basis for cold acclimation in Drosophila larvae

Low temperatures can be fatal to insects, but many species have evolved the ability to cold acclimate, thereby increasing their cold tolerance. It has been previously shown that Drosophila melanogaster larvae perform cold-evoked behaviors under the control of noxious cold-sensing neurons (nociceptors), but it is unknown how the nervous system might participate in cold tolerance. Herein, we describe cold-nociceptive behavior among 11 drosophilid species; we find that the predominant cold-evoked larval response is a head-to-tail contraction behavior, which is likely inherited from a common ancestor, but is unlikely to be protective. We therefore tested the hypothesis that cold nociception functions to protect larvae by triggering cold acclimation. We found that Drosophila melanogaster Class III nociceptors are sensitized by and critical to cold acclimation and that cold acclimation can be optogenetically evoked, sans cold. Collectively, these findings demonstrate that cold nociception constitutes a peripheral neural basis for Drosophila larval cold acclimation.

There is life beyond the statistical significance

This article challenges the "tyranny of P-value" and promote more valuable and applicable interpretations of the results of research on health care delivery. We provide here solid arguments to retire statistical significance as the unique way to interpret results, after presenting the current state of the debate inside the scientific community. Instead, we promote reporting the much more informative confidence intervals and eventually adding exact P-values. We also provide some clues to integrate statistical and clinical significance by referring to minimal important differences and integrating the effect size of an intervention and the certainty of evidence ideally using the GRADE approach. We have argued against interpreting or reporting results as statistically significant or statistically non-significant. We recommend showing important clinical benefits with their confidence intervals in cases of point estimates compatible with results benefits and even important harms. It seems fair to report the point estimate and the more likely values along with a very clear statement of the implications of extremes of the intervals. We recommend drawing conclusions, considering the multiple factors besides P-values such as certainty of the evidence for each outcome, net benefit, economic considerations and values and preferences. We use several examples and figures to illustrate different scenarios and further suggest a wording to standardize the reporting. Several statistical measures have a role in the scientific communication of studies, but it is time to understand that there is life beyond the statistical significance. There is a great opportunity for improvement towards a more complete interpretation and to a more standardized reporting.

Drug Use and Cancer Risk: A Drug-Wide Association Study (DWAS) in Norway

BACKGROUND

Population-based pharmaco-epidemiologic studies are used to assess postmarketing drug safety and discover beneficial effects of off-label drug use. We conducted a drug-wide association study (DWAS) to screen for associations between prescription drugs and cancer risk.

METHODS

This registry-based, nested case-control study, 1:10 matched on age, sex, and date of diagnosis of cases, comprises approximately 2 million Norwegian residents, including their drug history from 2004 to 2014. We evaluated the association between prescribed drugs, categorized according to the anatomical therapeutic chemical (ATC) classification system, and the risk of the 15 most common cancer types, overall and by histology. We used stratified Cox regression, adjusted for other drug use, comorbidity, county, and parity, and explored dose-response trends.

RESULTS

We found 145 associations among 1,230 drug-cancer combinations on the ATC2-level and 77 of 8,130 on the ATC4-level. Results for all drug-cancer combinations are presented in this article and an online tool (https://pharmacoepi.shinyapps.io/drugwas/). Some associations have been previously reported, that is, menopausal hormones and breast cancer risk, or are likely confounded, that is, chronic obstructive pulmonary diseases and lung cancer risk. Other associations were novel, that is, inverse association between proton pump inhibitors and melanoma risk, and carcinogenic association of propulsives and lung cancer risk.

CONCLUSIONS

This study confirmed previously reported associations and generated new hypotheses on possible carcinogenic or chemopreventive effects of prescription drugs. Results from this type of explorative approach need to be validated in tailored epidemiologic and preclinical studies.

IMPACT

DWAS studies are robust and important tools to define new drug-cancer hypotheses.See related commentary by Wang and Gadalla, p. 597.

Factors that impact fragility index and their visualizations

RATIONALE AIMS AND OBJECTIVES

As the recent literature has growing concerns about research replicability and the misuse and misconception of P-values, the fragility index (FI) has been an attractive measure to assess the robustness (or fragility) of clinical study results with binary outcomes. It is defined as the minimum number of event status modifications that can alter a study result's statistical significance (or non-significance). Owing to its intuitive concept, the FI has been applied to assess the fragility of clinical studies of various specialties. However, the FI may be limited in certain settings. As a relatively new measure, more work is needed to examine its properties.

METHODS

This article explores several factors that may impact the derivation of the FI, including how event status is modified and the impact of significance levels. Moreover, we propose novel methods to visualize the fragility of a study's result. These factors and methods are illustrated using worked examples of artificial datasets. Randomized controlled trials on antidepressant drugs are also used to evaluate their real-world performance.

RESULTS

The FI depends on the treatment arm(s) in which event status is modified, whether the original study result is significant, the statistical method used for calculating the P-value, and the threshold for determining statistical significance. Also, the proposed visualization methods can clearly demonstrate a study result's fragility, which may be useful supplements to the single value of the FI.

CONCLUSIONS

Our findings may help clinicians properly use the FI and appraise the reliability of a study's conclusion.