Exploratory reports: A new article type for Cortex

Demystifying Open Science in health psychology and behavioral medicine: a practical guide to Registered Reports and Data Notes

Open Science practices are integral to increasing transparency, reproducibility, and accessibility of research in health psychology and behavioral medicine. Drives to facilitate Open Science practices are becoming increasingly evident in journal editorial policies, including the establishment of new paper formats such as Registered Reports and Data Notes. This paper provides: (i) an overview of the current state of Open Science policies within health psychology and behavioral medicine, (ii) a call for submissions to an Article Collection of Registered Reports and Data Notes as new paper formats within the journal of Health Psychology & Behavioral Medicine, (iii) an overview of Registered Reports and Data Notes, and (iv) practical considerations for authors and reviewers of Registered Reports and Data Notes.

Registered report: Survey on attitudes and experiences regarding preregistration in psychological research

BACKGROUND

Preregistration, the open science practice of specifying and registering details of a planned study prior to knowing the data, increases the transparency and reproducibility of research. Large-scale replication attempts for psychological results yielded shockingly low success rates and contributed to an increasing demand for open science practices among psychologists. However, preregistering one's studies is still not the norm in the field. Here, we conducted a study to explore possible reasons for this discrepancy.

METHODS

In a mixed-methods approach, we conducted an online survey assessing attitudes, motivations, and perceived obstacles with respect to preregistration. Respondents (N = 289) were psychological researchers that were recruited through their publications on Web of Science, PubMed, PSYNDEX, and PsycInfo, and preregistrations on OSF Registries. Based on the theory of planned behavior, we predicted that positive attitudes (moderated by the perceived importance of preregistration) as well as a favorable subjective norm and higher perceived behavioral control positively influence researchers' intention to preregister (directional hypothesis 1). Furthermore, we expected an influence of research experience on attitudes and perceived motivations and obstacles regarding preregistration (non-directional hypothesis 2). We analyzed these hypotheses with multiple regression models and included preregistration experience as a control variable.

RESULTS

Researchers' attitudes, subjective norms, perceived behavioral control, and the perceived importance of preregistration significantly predicted researchers' intention to use preregistration in the future (see hypothesis 1). Research experience influenced both researchers' attitudes and their perception of motivations to preregister, but not the perception of obstacles (see hypothesis 2). Descriptive reports on researchers' attitudes, motivations and obstacles regarding preregistration are provided.

DISCUSSION

Many researchers had already preregistered and had a rather positive attitude toward preregistration. Nevertheless, several obstacles were identified that may be addressed to improve and foster preregistration.

Reducing bias, increasing transparency and calibrating confidence with preregistration

Flexibility in the design, analysis and interpretation of scientific studies creates a multiplicity of possible research outcomes. Scientists are granted considerable latitude to selectively use and report the hypotheses, variables and analyses that create the most positive, coherent and attractive story while suppressing those that are negative or inconvenient. This creates a risk of bias that can lead to scientists fooling themselves and fooling others. Preregistration involves declaring a research plan (for example, hypotheses, design and statistical analyses) in a public registry before the research outcomes are known. Preregistration (1) reduces the risk of bias by encouraging outcome-independent decision-making and (2) increases transparency, enabling others to assess the risk of bias and calibrate their confidence in research outcomes. In this Perspective, we briefly review the historical evolution of preregistration in medicine, psychology and other domains, clarify its pragmatic functions, discuss relevant meta-research, and provide recommendations for scientists and journal editors.

Transparent, Open, and Reproducible Prevention Science

The field of prevention science aims to understand societal problems, identify effective interventions, and translate scientific evidence into policy and practice. There is growing interest among prevention scientists in the potential for transparency, openness, and reproducibility to facilitate this mission by providing opportunities to align scientific practice with scientific ideals, accelerate scientific discovery, and broaden access to scientific knowledge. The overarching goal of this manuscript is to serve as a primer introducing and providing an overview of open science for prevention researchers. In this paper, we discuss factors motivating interest in transparency and reproducibility, research practices associated with open science, and stakeholders engaged in and impacted by open science reform efforts. In addition, we discuss how and why different types of prevention research could incorporate open science practices, as well as ways that prevention science tools and methods could be leveraged to advance the wider open science movement. To promote further discussion, we conclude with potential reservations and challenges for the field of prevention science to address as it transitions to greater transparency, openness, and reproducibility. Throughout, we identify activities that aim to strengthen the reliability and efficiency of prevention science, facilitate access to its products and outputs, and promote collaborative and inclusive participation in research activities. By embracing principles of transparency, openness, and reproducibility, prevention science can better achieve its mission to advance evidence-based solutions to promote individual and collective well-being.

Is It Just Face Blindness? Exploring Developmental Comorbidity in Individuals with Self-Reported Developmental Prosopagnosia

Developmental prosopagnosia (DP)—or ‘face blindness’—refers to life-long problems with facial recognition in the absence of brain injury. We know that neurodevelopmental disorders tend to co-occur, and this study aims to explore if individuals with self-reported DP also report indications of other neurodevelopmental disorders, deficits, or conditions (developmental comorbidity). In total, 115 individuals with self-reported DP participated in this online cross-sectional survey. Face recognition impairment was measured with a validated self-report instrument. Indications of difficulties with navigation, math, reading, or spelling were measured with a tailored questionnaire using items from published sources. Additional diagnoses were measured with direct questions. We also included open-ended questions about cognitive strengths and difficulties. Results: Overall, 57% reported at minimum one developmental comorbidity of interest, with most reflecting specific cognitive impairment (e.g., in memory or object recognition) rather than diagnostic categories (e.g., ADHD, dyslexia). Interestingly, many participants reported cognitive skills or strengths within the same domains that others reported impairment, indicating a diverse pattern of cognitive strengths and difficulties in this sample. The frequency and diversity of self-reported developmental comorbidity suggests that face recognition could be important to consider in future investigations of neurodevelopmental comorbidity patterns.

Protecting against researcher bias in secondary data analysis: challenges and potential solutions

Analysis of secondary data sources (such as cohort studies, survey data, and administrative records) has the potential to provide answers to science and society's most pressing questions. However, researcher biases can lead to questionable research practices in secondary data analysis, which can distort the evidence base. While pre-registration can help to protect against researcher biases, it presents challenges for secondary data analysis. In this article, we describe these challenges and propose novel solutions and alternative approaches. Proposed solutions include approaches to (1) address bias linked to prior knowledge of the data, (2) enable pre-registration of non-hypothesis-driven research, (3) help ensure that pre-registered analyses will be appropriate for the data, and (4) address difficulties arising from reduced analytic flexibility in pre-registration. For each solution, we provide guidance on implementation for researchers and data guardians. The adoption of these practices can help to protect against researcher bias in secondary data analysis, to improve the robustness of research based on existing data.

The past, present and future of Registered Reports

Registered Reports are a form of empirical publication in which study proposals are peer reviewed and pre-accepted before research is undertaken. By deciding which articles are published based on the question, theory and methods, Registered Reports offer a remedy for a range of reporting and publication biases. Here, we reflect on the history, progress and future prospects of the Registered Reports initiative and offer practical guidance for authors, reviewers and editors. We review early evidence that Registered Reports are working as intended, while at the same time acknowledging that they are not a universal solution for irreproducibility. We also consider how the policies and practices surrounding Registered Reports are changing, or must change in the future, to address limitations and adapt to new challenges. We conclude that Registered Reports are promoting reproducibility, transparency and self-correction across disciplines and may help reshape how society evaluates research and researchers.

Why Hypothesis Testers Should Spend Less Time Testing Hypotheses

For almost half a century, Paul Meehl educated psychologists about how the mindless use of null-hypothesis significance tests made research on theories in the social sciences basically uninterpretable. In response to the replication crisis, reforms in psychology have focused on formalizing procedures for testing hypotheses. These reforms were necessary and influential. However, as an unexpected consequence, psychological scientists have begun to realize that they may not be ready to test hypotheses. Forcing researchers to prematurely test hypotheses before they have established a sound "derivation chain" between test and theory is counterproductive. Instead, various nonconfirmatory research activities should be used to obtain the inputs necessary to make hypothesis tests informative. Before testing hypotheses, researchers should spend more time forming concepts, developing valid measures, establishing the causal relationships between concepts and the functional form of those relationships, and identifying boundary conditions and auxiliary assumptions. Providing these inputs should be recognized and incentivized as a crucial goal in itself. In this article, we discuss how shifting the focus to nonconfirmatory research can tie together many loose ends of psychology's reform movement and help us to develop strong, testable theories, as Paul Meehl urged.

EEG power spectral dynamics associated with listening in adverse conditions

Adverse listening conditions increase the demand on cognitive resources needed for speech comprehension. In an exploratory study, we aimed to identify independent power spectral features in the EEG useful for studying the cognitive processes involved in this effortful listening. Listeners performed the coordinate response measure task with a single-talker masker at a 0-dB signal-to-noise ratio. Sounds were left unfiltered or degraded with low-pass filtering. Independent component analysis (ICA) was used to identify independent components (ICs) in the EEG data, the power spectral dynamics of which were then analyzed. Frontal midline theta, left frontal, right frontal, left mu, right mu, left temporal, parietal, left occipital, central occipital, and right occipital clusters of ICs were identified. All IC clusters showed some significant listening-related changes in their power spectrum. This included sustained theta enhancements, gamma enhancements, alpha enhancements, alpha suppression, beta enhancements, and mu rhythm suppression. Several of these effects were absent or negligible using traditional channel analyses. Comparison of filtered to unfiltered speech revealed a stronger alpha suppression in the parietal and central occipital clusters of ICs for the filtered speech condition. This not only replicates recent findings showing greater alpha suppression as listening difficulty increases but also suggests that such alpha-band effects can stem from multiple cortical sources. We lay out the advantages of the ICA approach over the restrictive analyses that have been used as of late in the study of listening effort. We also make suggestions for moving into hypothesis-driven studies regarding the power spectral features that were revealed.

Moving Sport and Exercise Science Forward: A Call for the Adoption of More Transparent Research Practices

The primary means of disseminating sport and exercise science research is currently through journal articles. However, not all studies, especially those with null findings, make it to formal publication. This publication bias towards positive findings may contribute to questionable research practices. Preregistration is a solution to prevent the publication of distorted evidence resulting from this system. This process asks authors to register their hypotheses and methods before data collection on a publicly available repository or by submitting a Registered Report. In the Registered Report format, authors submit a stage 1 manuscript to a participating journal that includes an introduction, methods, and any pilot data indicating the exploratory or confirmatory nature of the study. After a stage 1 peer review, the manuscript can then be offered in-principle acceptance, rejected, or sent back for revisions to improve the quality of the study. If accepted, the project is guaranteed publication, assuming the authors follow the data collection and analysis protocol. After data collection, authors re-submit a stage 2 manuscript that includes the results and discussion, and the study is evaluated on clarity and conformity with the planned analysis. In its final form, Registered Reports appear almost identical to a typical publication, but give readers confidence that the hypotheses and main analyses are less susceptible to bias from questionable research practices. From this perspective, we argue that inclusion of Registered Reports by researchers and journals will improve the transparency, replicability, and trust in sport and exercise science research. The preprint version of this work is available on SportR[Formula: see text]iv: https://osf.io/preprints/sportrxiv/fxe7a/.

The case for formal methodology in scientific reform

Current attempts at methodological reform in sciences come in response to an overall lack of rigor in methodological and scientific practices in experimental sciences. However, most methodological reform attempts suffer from similar mistakes and over-generalizations to the ones they aim to address. We argue that this can be attributed in part to lack of formalism and first principles. Considering the costs of allowing false claims to become canonized, we argue for formal statistical rigor and scientific nuance in methodological reform. To attain this rigor and nuance, we propose a five-step formal approach for solving methodological problems. To illustrate the use and benefits of such formalism, we present a formal statistical analysis of three popular claims in the metascientific literature: (i) that reproducibility is the cornerstone of science; (ii) that data must not be used twice in any analysis; and (iii) that exploratory projects imply poor statistical practice. We show how our formal approach can inform and shape debates about such methodological claims.

Cultural differences in food detection

The ability to detect food plays an indispensable role in our survival and wellbeing. Previous psychological studies have revealed that food is detected more rapidly than non-food items. However, whether the detection of food could be modulated by cultural factors remains unknown. We investigated this issue in the present study using a visual search paradigm with Polish and Japanese participants. Photographs of international fast food, domestic Japanese food, or kitchen tools were presented alongside images of non-food distractors (cars). Participants were asked to judge whether the stimuli were all identical or not. The reaction time data showed that participants from both cultures detected food more rapidly than kitchen tools. Japanese participants detected fast food more rapidly than Japanese food, whereas Polish participants did not display such differences between food types. These results suggest that rapid detection of food is universal, but is modulated by cultural experiences.

Transcranial direct current stimulation: a roadmap for research, from mechanism of action to clinical implementation

Transcranial direct current stimulation (tDCS) is a promising method for altering the function of neural systems, cognition, and behavior. Evidence is emerging that it can also influence psychiatric symptomatology, including major depression and schizophrenia. However, there are many open questions regarding how the method might have such an effect, and uncertainties surrounding its influence on neural activity, and human cognition and functioning. In the present critical review, we identify key priorities for future research into major depression and schizophrenia, including studies of the mechanism(s) of action of tDCS at the neuronal and systems levels, the establishment of the cognitive impact of tDCS, as well as investigations of the potential clinical efficacy of tDCS. We highlight areas of progress in each of these domains, including data that appear to favor an effect of tDCS on neural oscillations rather than spiking, and findings that tDCS administration to the prefrontal cortex during task training may be an effective way to enhance behavioral performance. Finally, we provide suggestions for further empirical study that will elucidate the impact of tDCS on brain and behavior, and may pave the way for efficacious clinical treatments for psychiatric disorders.

Open science challenges, benefits and tips in early career and beyond

The movement towards open science is a consequence of seemingly pervasive failures to replicate previous research. This transition comes with great benefits but also significant challenges that are likely to affect those who carry out the research, usually early career researchers (ECRs). Here, we describe key benefits, including reputational gains, increased chances of publication, and a broader increase in the reliability of research. The increased chances of publication are supported by exploratory analyses indicating null findings are substantially more likely to be published via open registered reports in comparison to more conventional methods. These benefits are balanced by challenges that we have encountered and that involve increased costs in terms of flexibility, time, and issues with the current incentive structure, all of which seem to affect ECRs acutely. Although there are major obstacles to the early adoption of open science, overall open science practices should benefit both the ECR and improve the quality of research. We review 3 benefits and 3 challenges and provide suggestions from the perspective of ECRs for moving towards open science practices, which we believe scientists and institutions at all levels would do well to consider.

Peer-Review Guidelines Promoting Replicability and Transparency in Psychological Science

More and more psychological researchers have come to appreciate the perils of common but poorly justified research practices and are rethinking commonly held standards for evaluating research. As this methodological reform expresses itself in psychological research, peer reviewers of such work must also adapt their practices to remain relevant. Reviewers of journal submissions wield considerable power to promote methodological reform, and thereby contribute to the advancement of a more robust psychological literature. We describe concrete practices that reviewers can use to encourage transparency, intellectual humility, and more valid assessments of the methods and statistics reported in articles.

Writing Empirical Articles: Transparency, Reproducibility, Clarity, and Memorability

This article provides recommendations for writing empirical journal articles that enable transparency, reproducibility, clarity, and memorability. Recommendations for transparency include preregistering methods, hypotheses, and analyses; submitting registered reports; distinguishing confirmation from exploration; and showing your warts. Recommendations for reproducibility include documenting methods and results fully and cohesively, by taking advantage of open--science tools, and citing sources responsibly. Recommendations for clarity include writing short paragraphs, composed of short sentences; writing comprehensive abstracts; and seeking feedback from a naive audience. Recommendations for memorability include writing narratively; embracing the hourglass shape of empirical articles; beginning articles with a hook; and synthesizing, rather than Mad Libbing, previous literature.

Peripersonal and extrapersonal visuospatial neglect in different frames of reference: A brain lesion-symptom mapping study

INTRODUCTION

Visuospatial neglect can occur in peripersonal and extrapersonal space. The dorsal visual pathway is hypothesized to be associated with peripersonal, and the ventral pathway with extrapersonal neglect. We aimed to evaluate neural substrates of peripersonal versus extrapersonal neglect, separately for egocentric and allocentric frames of reference.

METHODS

This was a retrospective study, including stroke patients admitted for inpatient rehabilitation. Approximately 1 month post-stroke onset, computerized cancellation (egocentric) and bisection tasks (egocentric and allocentric) were administered at 30 cm and 120 cm. We collected CT or MRI scans and performed voxel-based lesion-symptom mapping for the cancellation, and subtraction analyses for the line bisection task.

RESULTS

We included 98 patients for the cancellation and 129 for the bisection analyses. The right parahippocampal gyrus, hippocampus, and thalamus were associated with egocentric peripersonal neglect as measured with cancellation. These areas were also associated with extrapersonal neglect, together with the right superior parietal lobule, angular gyrus, supramarginal gyrus, lateral occipital cortex, planum temporale and superior temporal gyrus. Lesions in the right parietal, temporal and frontal areas were associated with both peripersonal and extrapersonal egocentric neglect as measured with bisection. For allocentric neglect no clear pattern of associated brain regions was observed.

DISCUSSION

We found right hemispheric anatomical correlates for peripersonal and extrapersonal neglect. However, no brain areas were uniquely associated with peripersonal neglect, meaning we could not conclusively verify the ventral/dorsal hypothesis. Several areas were uniquely associated with egocentric extrapersonal neglect, suggesting that these brain areas can be specifically involved in extrapersonal, but not in peripersonal, attention processes.