Pooling resources to enhance rigour in psychophysiological research: Insights from open science approaches to meta-analysis

Beyond single paradigms, pipelines, and outcomes: Embracing multiverse analyses in psychophysiology

Psychophysiological research is an inherently complex undertaking due to the nature of the data, and its analysis is characterized by many decision points that shape the final dataset and a study's findings. These decisions create a "multiverse" of possible outcomes, and each decision from study conceptualization to statistical analysis can lead to different results and interpretations. This review describes the concept of multiverse analyses, a methodological approach designed to understand the impact of different decisions on the robustness of a study's findings and interpretation. The emphasis is on transparently showcasing different reasonable approaches for constructing a final dataset and on highlighting the influence of various decision points, from experimental design to data processing and outcome selection. For example, the choice of an experimental task can significantly impact event-related brain potential (ERP) scores or skin conductance responses (SCRs), and different tasks might elicit unique variances in each measure. This review underscores the importance of transparently embracing the flexibility inherent in psychophysiological research and the potential consequences of not understanding the fragility or robustness of experimental findings. By navigating the intricate terrain of the psychophysiological multiverse, this review serves as an introduction, helping researchers to make informed decisions, improve the collective understanding of psychophysiological findings, and push the boundaries of the field.

Cautionary Observations Concerning the Introduction of Psychophysiological Biomarkers into Neuropsychiatric Practice

The combination of statistical learning technologies with large databases of psychophysiological data has appropriately generated enthusiastic interest in future clinical applicability. It is argued here that this enthusiasm should be tempered with the understanding that significant obstacles must be overcome before the systematic introduction of psychophysiological measures into neuropsychiatric practice becomes possible. The objective of this study is to identify challenges to this effort. The nonspecificity of psychophysiological measures complicates their use in diagnosis. Low test-retest reliability complicates use in longitudinal assessment, and quantitative psychophysiological measures can normalize in response to placebo intervention. Ten cautionary observations are introduced and, in some instances, possible directions for remediation are suggested.

Open Science in human electrophysiology

Open science practices are gaining momentum in psychophysiological research, but at the nascent stage of this special issue in the International Journal of Psychophysiology there was no systematic collection of resources to support the adoption of open science practices specific to studies of human electrophysiology (EEG). The purpose of this special issue was to gather and provide resources that identify the idiosyncratic considerations and implications of open science practices specifically for studies of human EEG and event-related potentials (ERPs). Papers also show the importance of promoting good scientific practices in the application of open science principles to EEG and ERPs. This introduction to the special issue provides a roadmap for identifying the resources necessary to begin and improve the application of open science principles to EEG and ERP research. We are optimistic that open science practices will help increase the robustness, rigor, and replicability of EEG research and ultimately become the norm in studies of EEG and ERPs.