Optimal detection of functional connectivity from high-dimensional EEG synchrony data

Influence of multiple hypothesis testing on reproducibility in neuroimaging research: A simulation study and Python-based software

BACKGROUND

Reproducibility of research findings has been recently questioned in many fields of science, including psychology and neurosciences. One factor influencing reproducibility is the simultaneous testing of multiple hypotheses, which entails false positive findings unless the analyzed p-values are carefully corrected. While this multiple testing problem is well known and studied, it continues to be both a theoretical and practical problem.

NEW METHOD

Here we assess reproducibility in simulated experiments in the context of multiple testing. We consider methods that control either the family-wise error rate (FWER) or false discovery rate (FDR), including techniques based on random field theory (RFT), cluster-mass based permutation testing, and adaptive FDR. Several classical methods are also considered. The performance of these methods is investigated under two different models.

RESULTS

We found that permutation testing is the most powerful method among the considered approaches to multiple testing, and that grouping hypotheses based on prior knowledge can improve power. We also found that emphasizing primary and follow-up studies equally produced most reproducible outcomes.

COMPARISON WITH EXISTING METHOD(S)

We have extended the use of two-group and separate-classes models for analyzing reproducibility and provide a new open-source software "MultiPy" for multiple hypothesis testing.

CONCLUSIONS

Our simulations suggest that performing strict corrections for multiple testing is not sufficient to improve reproducibility of neuroimaging experiments. The methods are freely available as a Python toolkit "MultiPy" and we aim this study to help in improving statistical data analysis practices and to assist in conducting power and reproducibility analyses for new experiments.

Statistical detection of EEG synchrony using empirical bayesian inference

There is growing interest in understanding how the brain utilizes synchronized oscillatory activity to integrate information across functionally connected regions. Computing phase-locking values (PLV) between EEG signals is a popular method for quantifying such synchronizations and elucidating their role in cognitive tasks. However, high-dimensionality in PLV data incurs a serious multiple testing problem. Standard multiple testing methods in neuroimaging research (e.g., false discovery rate, FDR) suffer severe loss of power, because they fail to exploit complex dependence structure between hypotheses that vary in spectral, temporal and spatial dimension. Previously, we showed that a hierarchical FDR and optimal discovery procedures could be effectively applied for PLV analysis to provide better power than FDR. In this article, we revisit the multiple comparison problem from a new Empirical Bayes perspective and propose the application of the local FDR method (locFDR; Efron, 2001) for PLV synchrony analysis to compute FDR as a posterior probability that an observed statistic belongs to a null hypothesis. We demonstrate the application of Efron's Empirical Bayes approach for PLV synchrony analysis for the first time. We use simulations to validate the specificity and sensitivity of locFDR and a real EEG dataset from a visual search study for experimental validation. We also compare locFDR with hierarchical FDR and optimal discovery procedures in both simulation and experimental analyses. Our simulation results showed that the locFDR can effectively control false positives without compromising on the power of PLV synchrony inference. Our results from the application locFDR on experiment data detected more significant discoveries than our previously proposed methods whereas the standard FDR method failed to detect any significant discoveries.

Damage to the Salience Network and interactions with the Default Mode Network

Interactions between the Salience Network (SN) and the Default Mode Network (DMN) are thought to be important for cognitive control. However, evidence for a causal relationship between the networks is limited. Previously, we have reported that traumatic damage to white matter tracts within the SN predicts abnormal DMN function. Here we investigate the effect of this damage on network interactions that accompany changing motor control. We initially used fMRI of the Stop Signal Task to study response inhibition in humans. In healthy subjects, functional connectivity (FC) between the right anterior insula (rAI), a key node of the SN, and the DMN transiently increased during stopping. This change in FC was not seen in a group of traumatic brain injury (TBI) patients with impaired cognitive control. Furthermore, the amount of SN tract damage negatively correlated with FC between the networks. We confirmed these findings in a second group of TBI patients. Here, switching rather than inhibiting a motor response: (1) was accompanied by a similar increase in network FC in healthy controls; (2) was not seen in TBI patients; and (3) tract damage after TBI again correlated with FC breakdown. This shows that coupling between the rAI and DMN increases with cognitive control and that damage within the SN impairs this dynamic network interaction. This work provides compelling evidence for a model of cognitive control where the SN is involved in the attentional capture of salient external stimuli and signals the DMN to reduce its activity when attention is externally focused.