Seeing more than the Tip of the Iceberg: Approaches to Subthreshold Effects in Functional Magnetic Resonance Imaging of the Brain

Many functional magnetic resonance imaging (fMRI) studies and presurgical mapping applications rely on mass-univariate inference with subsequent multiple comparison correction. Statistical results are frequently visualized as thresholded statistical maps. This approach has inherent limitations including the risk of drawing overly-selective conclusions based only on selective results passing such thresholds. This article gives an overview of both established and newly emerging scientific approaches to supplement such conventional analyses by incorporating information about subthreshold effects with the aim to improve interpretation of findings or leverage a wider array of information. Topics covered include neuroimaging data visualization, p-value histogram analysis and the related Higher Criticism approach for detecting rare and weak effects. Further examples from multivariate analyses and dedicated Bayesian approaches are provided.

"Trend" Statement Use in the Orthopaedic Literature

INTRODUCTION

For research to effectively guide clinical decision making, appropriate interpretation of data is paramount. The P -value is a useful tool for guiding the interpretation of data. However, despite its utility, the P -value is not without limitations. Of particular concern is the use of "trend statements" to describe non-statistically significant findings, a practice which introduces subjectivity and variability into data interpretation and can lead to the drawing of undue conclusions.

METHODS

An audit of original research articles published from January 2022 to December 2022 in four high-impact orthopaedic journals was conducted. The selected journals were queried to identify instances in which a non-statistically significant result was labeled as a "trend." The use of trend statements and associated information was recorded and analyzed.

RESULTS

One thousand two hundred sixty articles were included in the analysis. 81 articles (6.4%) included a trend statement to describe a non-statistically significant result. Only two articles (2.5%) formally defined what constituted a trend. In 28.8% of cases, the associated P -value was > 0.10.

DISCUSSION

Trend statements are used to describe non-statistically significant findings with moderate frequency in the orthopaedic literature. Given the potentially misleading effects of trend statements, efforts should be made to mitigate their use. If trend statements are to be used, attention should be paid to defining what constitutes a "trend", explicitly acknowledging the lack of statistical significance of the finding to which the trend statement refers, and avoiding drawing undue conclusions from non-statistically significant data.

Misinterpretations of Significance Testing Results Near the P-Value Threshold in the Urologic Literature

Background The outcome of a statistical test is to accept or reject a null hypothesis. Reporting a metric as "trending toward significance" is a misinterpretation of the p-value. Studies highlighting the prevalence of statistical errors in the urologic literature remain scarce. We evaluated abstracts from 15 urology journals published within the years 2000-2021 and provided a quantitative measure of a common statistical mistake-misconstruing the function of null hypothesis testing by reporting "a trend toward significance." Materials and methods We performed an audit of 15 urology journals, looking at articles published from January 1, 2000, to January 1, 2022. A word recognition function in Microsoft Excel was utilized to identify the use of the word "trend" in the abstracts. Each use of the word "trend" was manually investigated by two authors to determine whether it was improperly used in describing non-statistically significant data as trending toward significance. Statistics and data analysis were performed using Python libraries: pandas, scipy.stats, and seaborn. Results This study included 101,134 abstracts from 15 urology journals. Within those abstracts, the word "trend" was used 2,509 times, 572 uses of which were describing non-statistically significant data as trending toward significance. There was a statistically significant difference in the rate of errors between the 15 journals (p < 0.01). The highest rate of improper use of the word "trend" was found in Bladder Cancer with a rate of 1.6% (p < 0.01) of articles. The lowest rate of improper use was found in European Urology, with a rate of 0.3% (p < 0.01). Our analysis found a moderate correlation between the number of articles published and the number of misuses of the word "trend" within each journal and across all journals every year (r = 0.61 and 0.70, respectively). Conclusion The overall rate of p-value misinterpretation never exceeded 2% of articles in each journal. There is significance in the difference in misinterpretation rates between the different journals. Authors' utilization of the word "trend" describing non-significant p-values as being near significant should be used with caution.

Efficacy of levosimendan infusion in patients undergoing a left ventricular assist device implant in a propensity score matched analysis of the EUROMACS registry-the Euro LEVO-LVAD study

OBJECTIVES

Early right-sided heart failure (RHF) was seen in 22% of recipients of a left ventricular assist device (LVAD) in the European Registry for Patients with Mechanical Circulatory Support (EUROMACS). However, the optimal treatment of post-LVAD RHF is not well known. Levosimendan has proven to be effective in patients with cardiogenic shock and in those with end-stage heart failure. We sought to evaluate the efficacy of levosimendan on post-LVAD RHF and 30-day and 1-year mortality.

METHODS

The EUROMACS Registry was used to identify adults with mainstream continuous-flow LVAD implants who were treated with preoperative levosimendan compared to a propensity matched control cohort.

RESULTS

In total, 3661 patients received mainstream LVAD, of which 399 (11%) were treated with levosimendan pre-LVAD. Patients given levosimendan had a higher EUROMACS RHF score [4 (2- 5.5) vs 2 (2- 4); P < 0.001], received more right ventricular assist devices (RVAD) [32 (8%) vs 178 (5.5%); P = 0.038] and stayed longer in the intensive care unit post-LVAD implant [19 (8-35) vs 11(5-25); P < 0.001]. Yet, there was no significant difference in the rate of RHF, 30-day, or 1-year mortality. Also, in the matched cohort (357 patients taking levosimendan compared to an average of 622 controls across 20 imputations), we found no evidence for a difference in postoperative severe RHF, RVAD implant rate, length of stay in the intensive care unit or 30-day and 1-year mortality.

CONCLUSIONS

In this analysis of the EUROMACS registry, we found no evidence for an association between levosimendan and early RHF or death, albeit patients taking levosimendan had much higher risk profiles. For a definitive conclusion, a multicentre, randomized study is warranted.

The trends for the "trend toward significance" in the pediatric literature

Purpose This study is to examine whether the term "trend toward statistical significance" is used to describe statistically nonsignificant results in biomedical literature. We examined articles published in five high-impact pediatric journals, including The Lancet Child & Adolescent Health, The Journal of Pediatrics, Early Human Development, Frontiers in Pediatrics, and BMC Pediatrics to identify manuscripts where a "trend" was used to describe a statistically nonsignificant result, from January 2020 to December 2021, and, furthermore, for The Journal of Pediatrics, Early Human Development, and BMC Pediatrics from January 2010 to December 2011. We detected that a "trend toward significance" was used to describe a statistically nonsignificant result at least once in 146 articles (2.7%) during the period between 2020 and 2021 and in 97 articles (4.0%) during the period between 2010 and 2011. We found no significant difference in the proportion of published articles with inappropriate use of "trend" across journals belonging to the first quartile of impact compared to the second quartile or across journals publishing under the subscription model or open access policy compared to journals publishing solely under the open access policy, in any period. The overall proportion of the inappropriate use of "trend" declined significantly between 2010 and 2011 to 2020 and 2021 (p = 0.002, RR 0.66 95% CI 0.51-0.86).

CONCLUSION

"Trend" statements were sporadically used to describe statistically nonsignificant results across pediatric literature. The inappropriate use of "trend" to describe almost significant differences could be misleading, and "trend" should be reserved only when a specific statistical test for trend has been performed, or in relation to appropriate scientific definitions.

WHAT IS KNOWN

•Previously, researchers have reported inappropriate use of "trend" in articles across anaesthesia or major oncology journals. •In many cases, hypothesized results that are close but not lower than the statistical significance threshold are emphasized as "almost" significant.

WHAT IS NEW

•"Trend" statements were sporadically used to describe statistically nonsignificant results across pediatric literature. •Inappropriate use of  "trend" was similar in journals with a subscription model compared to those having an open access policy and decreased within a 10-year period.

Haemostatic profile of children with nephrotic syndrome attending University of Nigeria Teaching Hospital Ituku-Ozalla, Nigeria

Background

Haemostatic derangements are thought to be due to an imbalance between hepatic synthesis of pro-coagulants and urinary losses of anticoagulants.

Objectives

This study evaluated the coagulation profile of Nigerian children with nephrotic syndrome and examined the relationship between coagulation variables, disease state and steroid responsiveness.

Methods

A cross- sectional hospital based study on evaluation of coagulation profile of children with nephrotic syndrome compared with their age- and gender- matched controls.

Results

The median fibrinogen level in subjects and controls was the same (2.9 g/L). Sixteen of 46 (35%) children with nephrotic syndrome had hyperfibrinogenaemia. The median fibrinogen level of children in remission was 2.3 g/L and differed significantly when compared with those of children in relapse (p = 0.001). The median APTT of children with nephrotic syndrome was 45.0 s and differed significantly compared with those of controls (42.0 s) (p value = 0.02). The median prothrombin time in children with and without nephrotic syndrome were 12.0 and 13.0 s respectively, (p = 0.004). About 90% of children with nephrotic syndrome had INR within reference range. Thrombocytosis was found in 15% of children with nephrotic syndrome. The median platelet count in children with new disease was 432 × 10³cells/mm³ and differed significantly when compared with those of controls (p = 0.01).

INR was significantly shorter in children with steroid resistant nephrotic syndrome (SRNS) (median 0.8 s; IQR 0.8 -0.9 s) compared with controls (median 1.0 s; IQR 1.0 -1.1 s) (p = 0.01). Steroid sensitivity was the strongest predictor of remission in children with nephrotic syndrome; steroid sensitive patients were 30 times more likely to be in remission than in relapse (OR 30.03; CI 2.01 – 448.04).

Conclusion

This study shows that the haemostatic derangements in childhood nephrotic involve mostly fibrinogen, APTT, PT, INR and platelet counts. Antithrombin levels are largely unaffected. Variations in fibrinogen, APTT, PT and INR values may be due to the heterogeneous nature of the disease.

Misinterpretations of Null Hypothesis Significance Testing Results Near the P-value Threshold: An Estimation of The Prevalence of Such a Statistical Error In The Neurosurgical Literature

INTRODUCTION

Although statistical errors have been shown to be prevalent in the medical literature there has been a paucity of studies focusing on the prevalence of such errors in neurosurgery. In this article, we audit the contemporary neurosurgical literature for a common statistical mistake, namely, misinterpretations of null hypothesis significance testing results near the p-value threshold by conveying the idea of a "trend".

METHODS

PubMed/Medline was used to identify all articles published in six major neurosurgical journals between 2000 and 2020. The abstracts of these articles were extracted and scrutinized to determine when the word "trend" was used to express "near-statistical significance".

RESULTS

A total of 45,244 articles were included. Of those 461 (1.02% [0.86-1.18]) employed the word "trend" to indicate near statistical significance, a total of 3.8 [2.93-4.75] errors per issue per year. The error under study occurred more frequently in JNS Spine and less frequently in Acta Neurochirurgica (p=0.007). On an annual basis, there was no linear correlation between the total number of papers published per journal and the number of errors in that journal (r=0.34, p = 0.01).

CONCLUSION

Misinterpretations of null hypothesis significance testing results near the p-value threshold is present in at least 1% of the neurosurgical literature. While we believe that most statistical errors in medicine in general and neurosurgery in particular may be unintentional in nature, additional measures should be enacted to prevent the subsequent adoption of such methodological practices amongst future researchers.

Misleading medical literature: An observational study

OBJECTIVE

Language that implies a conclusion not supported by the evidence is common in the medical literature. The hypothesis of the present study was that medical journal publications are more likely to use misleading language for the interpretation of a demonstrated null (i.e. chance or not statistically significant) effect than a demonstrated real (i.e. statistically significant) effect.

METHODS

This was an observational study of the medical literature with a systematic sampling method. Articles published in The Journal of the American Medical Association, The Lancet and The New England Journal of Medicine over the last two decades were eligible. The language used around the P-value was assessed for misleadingness (i.e. either suggesting an effect existed when a real effect did not exist or vice versa).

RESULTS

There were 228 unique manuscripts examined, containing 400 statements interpreting a P-value proximate to 0.05. The P-value was between 0.036 and 0.050 for 303 (75.8%) statements and between 0.050 and 0.064 for 97 (24.3%) statements. Forty-four (11%) of the statements were misleading. There were 40 (41.2%) false-positive sentences, implying statistical significance when the P-value was >0.05, and four (1.3%) false-negative sentences, implying no statistical significance when the P-value <0.05 (relative risk 31.2; 95% confidence interval 11.5-85.1; P < 0.0001). The proportion of included manuscripts containing at least one misleading sentence was 16.2% (95% confidence interval 12.0-21.6).

CONCLUSIONS

Among a random selection of sentences in prestigious journals describing P-values close to 0.05, 1 in 10 are misleading (n = 44, 11%) and this is more prevalent when the P-values are above 0.05 compared to below 0.05. Caution is advised for researchers, clinicians and editors to align with the context and purpose of P-values.

Calhoun V, Grefkes C, Rost NS. Abnormal dynamic functional connectivity is linked to recovery after acute ischemic stroke

The aim of the current study was to explore the whole-brain dynamic functional connectivity patterns in acute ischemic stroke (AIS) patients and their relation to short and long-term stroke severity. We investigated resting-state functional MRI-based dynamic functional connectivity of 41 AIS patients two to five days after symptom onset. Re-occurring dynamic connectivity configurations were obtained using a sliding window approach and k-means clustering. We evaluated differences in dynamic patterns between three NIHSS-stroke severity defined groups (mildly, moderately, and severely affected patients). Furthermore, we built Bayesian hierarchical models to evaluate the predictive capacity of dynamic connectivity and examine the interrelation with clinical measures, such as white matter hyperintensity lesions. Finally, we established correlation analyses between dynamic connectivity and AIS severity as well as 90-day neurological recovery (ΔNIHSS). We identified three distinct dynamic connectivity configurations acutely post-stroke. More severely affected patients spent significantly more time in a configuration that was characterized by particularly strong connectivity and isolated processing of functional brain domains (three-level ANOVA: p < .05, post hoc t tests: p < .05, FDR-corrected). Configuration-specific time estimates possessed predictive capacity of stroke severity in addition to the one of clinical measures. Recovery, as indexed by the realized change of the NIHSS over time, was significantly linked to the dynamic connectivity between bilateral intraparietal lobule and left angular gyrus (Pearson's r = -.68, p = .003, FDR-corrected). Our findings demonstrate transiently increased isolated information processing in multiple functional domains in case of severe AIS. Dynamic connectivity involving default mode network components significantly correlated with recovery in the first 3 months poststroke.

The SSSPIN study-spin in studies of spin: meta-research analysis

OBJECTIVES

To identify and calculate the prevalence of spin in studies of spin.

DESIGN

Meta-research analysis (research on research).

SETTING

35 studies of spin in the scientific literature.

MAIN OUTCOME MEASURES

Spin, categorised as: reporting practices that distort the presentation and interpretation of results, creating misleading conclusions; discordance between results and their interpretation, with presentation of favourable conclusions that are not supported by the data or results; attribution of causality when study design does not support it; and over-interpretation or inappropriate extrapolation of results.

RESULTS

Five (14%) of 35 spin studies contained spin categorised as reporting practices that distort the presentation and interpretation of results (n=2) or categorised as over-interpretation or inappropriate extrapolation of results (n=3).

CONCLUSION

Spin occurs in research on spin. Although researchers on this topic should be sensitive to spinning their findings, our study does not undermine the need for rigorous interventions to reduce spin across various research fields.

CONCLUSION WITH SPIN

Our hypothesis that spin will be less prevalent in spin studies than in studies on other topics has been proven. Spin scholars are less likely to spin their conclusions than other researchers, and they should receive substantial resources to launch and test interventions to reduce spin and research waste in reporting.

Cardiorespiratory fitness and hippocampal volume predict faster episodic associative learning in older adults

Declining episodic memory is common among otherwise healthy older adults, in part due to negative effects of aging on hippocampal circuits. However, there is significant variability between individuals in severity of aging effects on the hippocampus and subsequent memory decline. Importantly, variability may be influenced by modifiable protective physiological factors such as cardiorespiratory fitness (CRF). More research is needed to better understand which aspects of cognition that decline with aging benefit most from CRF. The current study evaluated the relation of CRF with learning rate on the episodic associative learning (EAL) task, a task designed specifically to target hippocampal-dependent relational binding and to evaluate learning with repeated occurrences. Results show higher CRF was associated with faster learning rate. Larger hippocampal volume was also associated with faster learning rate, though hippocampal volume did not mediate the relationship between CRF and learning rate. Furthermore, to support the distinction between learning item relations and learning higher-order sequences, which declines with aging but is largely reliant on extra-hippocampal learning systems, we found learning rate on the EAL task was not related to motor sequence learning on the alternating serial reaction time task. Motor sequence learning was also not correlated with hippocampal volume. Thus, for the first time, we show that both higher CRF and larger hippocampal volume in healthy older adults are related to enhanced rate of relational memory acquisition.