Open science challenges, benefits and tips in early career and beyond

Moving Health Literacy Research and Practice towards a Vision of Equity, Precision and Transparency

Over the past two decades, health literacy research has gained increasing attention in global health initiatives to reduce health disparities. While it is well-documented that health literacy is associated with health outcomes, most findings are generated from cross-sectional data. Along with the increasing importance of health literacy in policy, there is a lack of specificity and transparency about how to improve health literacy in practice. In this study, we are calling for a shift of current research paradigms from judging health literacy levels towards observing how health literacy skills are developed over the life course and practised in the real world. This includes using a life-course approach, integrating the rationale of precision public health, applying open science practice, and promoting actionable knowledge translation strategies. We show how a greater appreciation for these paradigms promises to advance health literacy research and practice towards an equitable, precise, transparent, and actionable vision.

Improving open and rigorous science: ten key future research opportunities related to rigor, reproducibility, and transparency in scientific research

Background: As part of a coordinated effort to expand research activity around rigor, reproducibility, and transparency (RRT) across scientific disciplines, a team of investigators at the Indiana University School of Public Health-Bloomington hosted a workshop in October 2019 with international leaders to discuss key opportunities for RRT research. Objective: The workshop aimed to identify research priorities and opportunities related to RRT. Design: Over two-days, workshop attendees gave presentations and participated in three working groups: (1) Improving Education & Training in RRT, (2) Reducing Statistical Errors and Increasing Analytic Transparency, and (3) Looking Outward: Increasing Truthfulness and Accuracy of Research Communications. Following small-group discussions, the working groups presented their findings, and participants discussed the research opportunities identified. The investigators compiled a list of research priorities, which were circulated to all participants for feedback. Results: Participants identified the following priority research questions: (1) Can RRT-focused statistics and mathematical modeling courses improve statistics practice?; (2) Can specialized training in scientific writing improve transparency?; (3) Does modality (e.g. face to face, online) affect the efficacy RRT-related education?; (4) How can automated programs help identify errors more efficiently?; (5) What is the prevalence and impact of errors in scientific publications (e.g., analytic inconsistencies, statistical errors, and other objective errors)?; (6) Do error prevention workflows reduce errors?; (7) How do we encourage post-publication error correction?; (8) How does 'spin' in research communication affect stakeholder understanding and use of research evidence?; (9) Do tools to aid writing research reports increase comprehensiveness and clarity of research reports?; and (10) Is it possible to inculcate scientific values and norms related to truthful, rigorous, accurate, and comprehensive scientific reporting? Conclusion: Participants identified important and relatively unexplored questions related to improving RRT. This list may be useful to the scientific community and investigators seeking to advance meta-science (i.e. research on research).

Open synthesis and the coronavirus pandemic in 2020

•Open Science principles are vital for ensuring reproducibility, trust, and legacy. •Evidence synthesis is a vital means of summarizing research for decision-making. •Open Synthesis is the application of Open Science principles to evidence synthesis. •Open approaches to planning, conducting, and reporting synthesis have many benefits. •We call on the evidence synthesis community to embrace Open Synthesis.

Exploring factors that influence the practice of Open Science by early career health researchers: a mixed methods study

Background: There is a growing global movement towards open science and ensuring that health research is more transparent. It is vital that the researchers are adequately prepared for this research environment from early in their careers. However, the barriers and enablers to practicing open science for early career researchers (ECRs) have been explored to a limited extent. This study aimed to explore the views, experiences and factors influencing open science practices amongst ECRs working in health research. Methods: Semi-structured individual interviews were conducted with a convenience sample of ECRs working in health research. Participants also completed surveys regarding the factors influencing open science practices. Thematic analysis was used to analyse the qualitative data and descriptive statistical analyses were used to analyse survey data. Results: 14 ECRs participated. Two main themes were identified from interview data; Valuing Open Science and Creating a Culture for Open Science. Within 'Valuing Open Science', participants spoke about the conceptualisation of open science to be open across the entire research cycle, and important for producing better and more impactful research for patients and the public. Within 'Creating a Culture of Open Science' participants spoke about a number of factors influencing their practice of open science. These included cultural and academic pressures, the positives and negatives of increased accountability and transparency, and the need for more training and supporting resources to facilitate open science practices. Conclusion: ECRs see the importance of open science for beneficially impacting patient and public health but many feel that they are not fully supported to practice open science. Resources and supports including education and training are needed, as are better incentives for open science activities. Crucially, tangible engagement from institutions, funders and researchers is needed to facilitate the development of an open science culture.

Effective Self-Management for Early Career Researchers in the Natural and Life Sciences

Early career researchers (ECRs) are faced with a range of competing pressures in academia, making self-management key to building a successful career. The Organization for Human Brain Mapping undertook a group effort to gather helpful advice for ECRs in self-management.

The motivation for citizens' involvement in life sciences research is predicted by age and gender

Open Science is an umbrella term encompassing multiple concepts as open access to publications, open data, open education and citizen science that aim to make science more open and transparent. Citizen science, an important facet of Open Science, actively involves non-scientists in the research process, and can potentially be beneficial for multiple actors, such as scientists, citizens, policymakers and society in general. However, the reasons that motivate different segments of the public to participate in research are still understudied. Therefore, based on data gathered from a survey conducted in Czechia, Germany, Italy, Spain, Sweden, and the UK (N = 5,870), this study explores five types of incentives that can motivate individuals to become involved in life sciences research. The results demonstrate that men and younger individuals are more persuaded by extrinsic motives (external benefits or rewards), as compared with women and older people, who are driven by intrinsic motives (that originates from within an individual). This paper shows that specific strata of the population are differentially motivated to engage in research, thereby providing relevant knowledge for effectively designing public involvement activities that target various groups of the public in research projects.

Low availability of code in ecology: A call for urgent action

Access to analytical code is essential for transparent and reproducible research. We review the state of code availability in ecology using a random sample of 346 nonmolecular articles published between 2015 and 2019 under mandatory or encouraged code-sharing policies. Our results call for urgent action to increase code availability: only 27% of eligible articles were accompanied by code. In contrast, data were available for 79% of eligible articles, highlighting that code availability is an important limiting factor for computational reproducibility in ecology. Although the percentage of ecological journals with mandatory or encouraged code-sharing policies has increased considerably, from 15% in 2015 to 75% in 2020, our results show that code-sharing policies are not adhered to by most authors. We hope these results will encourage journals, institutions, funding agencies, and researchers to address this alarming situation.

Publication of the analytical code underlying a scientific study is increasingly expected or even mandated by journals, allowing others to reproduce the results. However, a survey of more than 300 recently published ecology papers finds the majority have no code publicly available, handicapping efforts to improve scientific transparency.

The Potential of Functional Near-Infrared Spectroscopy-Based Neurofeedback-A Systematic Review and Recommendations for Best Practice

Background: The effects of electroencephalography (EEG) and functional magnetic resonance imaging (fMRI)-neurofeedback on brain activation and behaviors have been studied extensively in the past. More recently, researchers have begun to investigate the effects of functional near-infrared spectroscopy-based neurofeedback (fNIRS-neurofeedback). FNIRS is a functional neuroimaging technique based on brain hemodynamics, which is easy to use, portable, inexpensive, and has reduced sensitivity to movement artifacts.

Method: We provide the first systematic review and database of fNIRS-neurofeedback studies, synthesizing findings from 22 peer-reviewed studies (including a total of N = 441 participants; 337 healthy, 104 patients). We (1) give a comprehensive overview of how fNIRS-neurofeedback training protocols were implemented, (2) review the online signal-processing methods used, (3) evaluate the quality of studies using pre-set methodological and reporting quality criteria and also present statistical sensitivity/power analyses, (4) investigate the effectiveness of fNIRS-neurofeedback in modulating brain activation, and (5) review its effectiveness in changing behavior in healthy and pathological populations.

Results and discussion: (1–2) Published studies are heterogeneous (e.g., neurofeedback targets, investigated populations, applied training protocols, and methods). (3) Large randomized controlled trials are still lacking. In view of the novelty of the field, the quality of the published studies is moderate. We identified room for improvement in reporting important information and statistical power to detect realistic effects. (4) Several studies show that people can regulate hemodynamic signals from cortical brain regions with fNIRS-neurofeedback and (5) these studies indicate the feasibility of modulating motor control and prefrontal brain functioning in healthy participants and ameliorating symptoms in clinical populations (stroke, ADHD, autism, and social anxiety). However, valid conclusions about specificity or potential clinical utility are premature.

Conclusion: Due to the advantages of practicability and relatively low cost, fNIRS-neurofeedback might provide a suitable and powerful alternative to EEG and fMRI neurofeedback and has great potential for clinical translation of neurofeedback. Together with more rigorous research and reporting practices, further methodological improvements may lead to a more solid understanding of fNIRS-neurofeedback. Future research will benefit from exploiting the advantages of fNIRS, which offers unique opportunities for neurofeedback research.

Mazor R, Theroux S, J. Gillett D, Lane B, Gearheart G. The importance of open science for biological assessment of aquatic environments

Open science principles that seek to improve science can effectively bridge the gap between researchers and environmental managers. However, widespread adoption has yet to gain traction for the development and application of bioassessment products. At the core of this philosophy is the concept that research should be reproducible and transparent, in addition to having long-term value through effective data preservation and sharing. In this article, we review core open science concepts that have recently been adopted in the ecological sciences and emphasize how adoption can benefit the field of bioassessment for both prescriptive condition assessments and proactive applications that inform environmental management. An example from the state of California demonstrates effective adoption of open science principles through data stewardship, reproducible research, and engagement of stakeholders with multimedia applications. We also discuss technical, sociocultural, and institutional challenges for adopting open science, including practical approaches for overcoming these hurdles in bioassessment applications.

Prediction of breeding regions for the desert locust Schistocerca gregaria in East Africa

Desert locust outbreak in East Africa is threatening livelihoods, food security, environment, and economic development in the region. The current magnitude of the desert locust invasion in East Africa is unprecedented and has not been witnessed for more than 70 years. Identifying the potential breeding sites of the pest is essential to carry out cost-effective and timely preventive measures before it inflicts significant damage. We accessed 9,134 desert locust occurrence records and applied a machine-learning algorithm to predict potential desert locust breeding sites in East Africa using key bio-climatic (temperature and rainfall) and edaphic (sand and moisture contents) factors. Ten days greenness maps from February 2020 to April 2020 were overlaid in model outputs to illustrate the temporal evolution of breeding site locations. This study demonstrated that vast areas of Kenya and Sudan, north eastern regions of Uganda, and south eastern and northern regions of South Sudan are at high risk of providing a conducive breeding environment for the desert locust. Our prediction results suggest that there is need to target these high-risk areas and strengthen ground surveillance to manage the pest in a timely, cost-effective, and environmentally friendly manner.

Transparency and open science principles in reporting guidelines in sleep research and chronobiology journals

Background: "Open science" is an umbrella term describing various aspects of transparent and open science practices. The adoption of practices at different levels of the scientific process (e.g., individual researchers, laboratories, institutions) has been rapidly changing the scientific research landscape in the past years, but their uptake differs from discipline to discipline. Here, we asked to what extent journals in the field of sleep research and chronobiology encourage or even require following transparent and open science principles in their author guidelines. Methods: We scored the author guidelines of a comprehensive set of 28 sleep and chronobiology journals, including the major outlets in the field, using the standardised Transparency and Openness (TOP) Factor. This instrument rates the extent to which journals encourage or require following various aspects of open science, including data citation, data transparency, analysis code transparency, materials transparency, design and analysis guidelines, study pre-registration, analysis plan pre-registration, replication, registered reports, and the use of open science badges. Results: Across the 28 journals, we find low values on the TOP Factor (median [25 th, 75 th percentile] 2.5 [1, 3], min. 0, max. 9, out of a total possible score of 28) in sleep research and chronobiology journals. Conclusions: Our findings suggest an opportunity for sleep research and chronobiology journals to further support the recent developments in transparent and open science by implementing transparency and openness principles in their guidelines and making adherence to them mandatory.

Graded fMRI Neurofeedback Training of Motor Imagery in Middle Cerebral Artery Stroke Patients: A Preregistered Proof-of-Concept Study

Ischemic stroke of the middle cerebral artery (MCA), a major brain vessel that supplies the primary motor and premotor cortex, is one of the most common causes for severe upper limb impairment. Currently available motor rehabilitation training largely lacks satisfying efficacy with over 70% of stroke survivors showing residual upper limb dysfunction. Motor imagery-based functional magnetic resonance imaging neurofeedback (fMRI-NF) has been suggested as a potential therapeutic technique to improve motor impairment in stroke survivors. In this preregistered proof-of-concept study (https://osf.io/y69jc/), we translated graded fMRI-NF training, a new paradigm that we have previously studied in healthy participants, to first-time MCA stroke survivors with residual mild to severe impairment of upper limb motor function. Neurofeedback was provided from the supplementary motor area (SMA) targeting two different neurofeedback target levels (low and high). We hypothesized that MCA stroke survivors will show (1) sustained SMA-region of interest (ROI) activation and (2) a difference in SMA-ROI activation between low and high neurofeedback conditions during graded fMRI-NF training. At the group level, we found only anecdotal evidence for these preregistered hypotheses. At the individual level, we found anecdotal to moderate evidence for the absence of the hypothesized graded effect for most subjects. These null findings are relevant for future attempts to employ fMRI-NF training in stroke survivors. The study introduces a Bayesian sequential sampling plan, which incorporates prior knowledge, yielding higher sensitivity. The sampling plan was preregistered together with a priori hypotheses and all planned analysis before data collection to address potential publication/researcher biases. Unforeseen difficulties in the translation of our paradigm to a clinical setting required some deviations from the preregistered protocol. We explicitly detail these changes, discuss the accompanied additional challenges that can arise in clinical neurofeedback studies, and formulate recommendations for how these can be addressed. Taken together, this work provides new insights about the feasibility of motor imagery-based graded fMRI-NF training in MCA stroke survivors and serves as a first example for comprehensive study preregistration of an (fMRI) neurofeedback experiment.

The Hong Kong Principles for assessing researchers: Fostering research integrity

For knowledge to benefit research and society, it must be trustworthy. Trustworthy research is robust, rigorous, and transparent at all stages of design, execution, and reporting. Assessment of researchers still rarely includes considerations related to trustworthiness, rigor, and transparency. We have developed the Hong Kong Principles (HKPs) as part of the 6th World Conference on Research Integrity with a specific focus on the need to drive research improvement through ensuring that researchers are explicitly recognized and rewarded for behaviors that strengthen research integrity. We present five principles: responsible research practices; transparent reporting; open science (open research); valuing a diversity of types of research; and recognizing all contributions to research and scholarly activity. For each principle, we provide a rationale for its inclusion and provide examples where these principles are already being adopted.

The Hong Kong Principles for assessing researchers: Fostering research integrity

For knowledge to benefit research and society, it must be trustworthy. Trustworthy research is robust, rigorous, and transparent at all stages of design, execution, and reporting. Assessment of researchers still rarely includes considerations related to trustworthiness, rigor, and transparency. We have developed the Hong Kong Principles (HKPs) as part of the 6th World Conference on Research Integrity with a specific focus on the need to drive research improvement through ensuring that researchers are explicitly recognized and rewarded for behaviors that strengthen research integrity. We present five principles: responsible research practices; transparent reporting; open science (open research); valuing a diversity of types of research; and recognizing all contributions to research and scholarly activity. For each principle, we provide a rationale for its inclusion and provide examples where these principles are already being adopted.

Reproducible research into human chemical communication by cues and pheromones: learning from psychology's renaissance

Despite the lack of evidence that the 'putative human pheromones' androstadienone and estratetraenol ever were pheromones, almost 60 studies have claimed 'significant' results. These are quite possibly false positives and can be best seen as potential examples of the 'reproducibility crisis', sadly common in the rest of the life and biomedical sciences, which has many instances of whole fields based on false positives. Experiments on the effects of olfactory cues on human behaviour are also at risk of false positives because they look for subtle effects but use small sample sizes. Research on human chemical communication, much of it falling within psychology, would benefit from vigorously adopting the proposals made by psychologists to enable better, more reliable science, with an emphasis on enhancing reproducibility. A key change is the adoption of study pre-registration and/or Registered Reports which will also reduce publication bias. As we are mammals, and chemical communication is important to other mammals, it is likely that chemical cues are important in our behaviour and that humans may have pheromones, but new approaches will be needed to reliably demonstrate them. This article is part of the Theo Murphy meeting issue 'Olfactory communication in humans'.

Millennial researchers in a metric-driven scholarly world: An international study

The study Investigates the attitudes and practices of early career researchers (ECRs) in regard to citation-based metrics and altmetrics, providing the findings in the light of what might be expected of the millennial generation and in the context of what we already know about researchers in today’s ‘culture of counting’ governed scholarly world. The data were gathered by means of an international survey, informed by a preceding, 3-year qualitative study of 120 ECRs from 7 countries, which obtained 1,600 responses. The main conclusions are: 1, citation indicators play a central and multi-purpose role in scholarly communications; 2, altmetrics are not so popular or widely used, but ECRs are waking up to some of their merits, most notably, discovering the extent to which their papers obtain traction and monitoring impact; 3, there is a strong likelihood that ECRs are going to have to grapple with both citation-based metrics and altmetrics, mainly in order to demonstrate research impact; 4, the Chinese are the most metric using nation, largely because of governmental regulations.

Improving the Rigor and Replicability of Applied Psychophysiology Research: Sample Size, Standardization, Transparency, and Preregistration

Scientific research across a number of areas, including applied psychophysiology, biofeedback, and neurofeedback, is facing considerable scrutiny for poor replication rates, high numbers of false-positive findings, and insufficient scientific rigor. There are many factors underlying this replication crisis in scientific research; yet incentives for more rigorous research practices at the institutional and editorial levels lag behind the need for improvement. The author provides examples of replication and rigor difficulties in scientific research with an eye toward psychophysiological research, including researcher flexibility in data analysis, “p-hacking,” insufficient sample sizes, and lack of availability and implementation of rigorous methodological and publication guidelines. Subsequently, the author highlights examples and opportunities for improvement, including decreasing researcher flexibility, reporting sample size information, increasing sample sizes through collaboration, improving reporting standards/following established guidelines for reporting psychophysiological data, and increasing adoption of preregistration and registered reports. The author concludes that the Association for Applied Psychophysiology and Biofeedback (AAPB) can improve clinical practice and perception of public and scientific credibility by implementing rigorous and transparent research practices with a focus on replicability and clear methodological and reporting techniques and standards.

Offloading items from memory: individual differences in cognitive offloading in a short-term memory task

Cognitive offloading refers to the act of reducing the mental processing requirements of a task through physical actions like writing down information or storing information on a cell phone or computer. Offloading can lead to improved performance on ongoing tasks with high cognitive demand, such as tasks where multiple pieces of information must be simultaneously maintained. However, less is known about why some individuals choose to engage in offloading and under what conditions they might choose to do so. In the present study, offloading behavior is investigated in a short-term memory task requiring memory for letters. The present study is a replication and extension of a previous study conducted by Risko and Dunn, and tests the new prediction that individuals with lower working memory capacity will be more likely to offload. Here, we find that offloading information confers a performance advantage over relying on internal memory stores, particularly at higher memory loads. However, we fail to observe that those with poorer memory abilities have a greater propensity for offloading or benefit more from it. Instead, our findings suggest that cognitive offloading may be a valid compensatory strategy to improve performance of memory-based tasks for individuals with a wide range of memory ability.

Enhancing scientific credibility: An open science strategy for the South African Journal of Industrial Psychology

Orientation: The purpose of this editorial was to provide an introduction and a general overview of the special issue on Open Science Practices: A Vision for the Future of SAJIP, as hosted in the 45th edition of the South African Journal of Industrial Psychology (SAJIP). Specifically, the aim was to provide a viable, practical and implementable strategy for enhancing the scientific credibility, transparency and international stature of SAJIP.

Correction: Open science challenges, benefits and tips in early career and beyond

[This corrects the article DOI: 10.1371/journal.pbio.3000246.].

Forensic science needs registered reports

The registered report (RR) format is rapidly being adopted by scientific researchers and journals. RRs flip the peer review process, with reviewers evaluating proposed methods, rather than the data and findings. Editors then accept or reject articles largely based on the pre-data collection review. Accordingly, RRs reduce the incentive for researchers to exaggerate their findings, and they make any data-driven changes to the methods and analysis more conspicuous. They also reduce publication bias, ensuring studies with null or otherwise unfavorable results are published. RRs are being used in many fields to improve research practices and increase confidence in study findings. The authors suggest RRs ought to be the default way in which validation studies are conducted and reported in the forensic sciences. They produce more reliable findings, advance criminal justice values, and will lead to several efficiencies in the research process.

Achieving Appropriate Model Transparency: Challenges and Potential Solutions for Making Value-Based Decisions in the United States

Transparency in decision modeling remains a topic of rigorous debate among healthcare stakeholders, given tensions between the potential benefits of external access during model development and the need to protect intellectual property and reward research investments. Strategies to increase decision model transparency by allowing direct external access to a model's structure, source code, and data can take on many forms but are bounded between the status quo and free publicly available open-source models. Importantly, some level of transparency already exists in terms of methods and other technical specifications for published models. The purpose of this paper is to delineate pertinent issues surrounding efforts to increase transparency via direct access to models and to offer key considerations for the field of health economics and outcomes research moving forward from a US academic perspective. Given the current environment faced by modelers in academic settings, expected benefits and challenges of allowing direct model access are discussed. The paper also includes suggestions for pathways toward increased transparency as well as an illustrative real-world example used in work with the Institute for Clinical and Economic Review to support assessments of the value of new health interventions. Potential options to increase transparency via direct model access during model development include adequate funding to support the additional effort required and mechanisms to maintain security of the underlying intellectual property. Ultimately, the appropriate level of transparency requires balancing the interests of several groups but, if done right, has the potential to improve models and better integrate them into healthcare priority setting and decision making in the US context.

Refining animal research: The Animal Study Registry

The Animal Study Registry (ASR; www.animalstudyregistry.org) was launched in January 2019 for preregistration of animal studies in order to increase transparency and reproducibility of bioscience research and to promote animal welfare. The registry is free of charge and is designed for exploratory and confirmatory studies within applied science as well as basic and preclinical research. The registration form helps scientists plan their study thoroughly by asking detailed questions concerning study design, methods, and statistics. With registration, the study automatically receives a digital object identifier (DOI) that marks it as intellectual property of the researcher. To accommodate the researchers concerns about theft of ideas, users can restrict the visibility of their registered studies for up to 5 years. The full content of the study becomes publicly accessible at the end of the embargo period. Because the platform is embedded in the infrastructure of the German Federal Government, continuity and data security are provided. By registering a study in the ASR, researchers can show their commitment to transparency and data quality to reviewers and editors, to third-party donors, and to the general public.

Preregistration of Analyses of Preexisting Data

The preregistration of a study's hypotheses, methods, and data-analyses steps is becoming a popular psychological research practice. To date, most of the discussion on study preregistration has focused on the preregistration of studies that include the collection of original data. However, much of the research in psychology relies on the (re-)analysis of preexisting data. Importantly, this type of studies is different from original studies as researchers cannot change major aspects of the study (e.g., experimental manipulations, sample size). Here, we provide arguments as to why it is useful to preregister analyses of preexisting data, discuss practical considerations, consider potential concerns, and introduce a preregistration template tailored for studies focused on the analyses of preexisting data. We argue that the preregistration of hypotheses and data-analyses for analyses of preexisting data is an important step towards more transparent psychological research.

Open Up - the Mission Statement of the Control of Impulsive Action (Ctrl-ImpAct) Lab on Open Science

The present paper is the mission statement of the Control of Impulsive Action (Ctrl-ImpAct) Lab regarding Open Science. As early-career researchers (ECRs) in the lab, we first state our personal motivation to conduct research based on the principles of Open Science. We then describe how we incorporate four specific Open Science practices (i.e., Open Methodology, Open Data, Open Source, and Open Access) into our scientific workflow. In more detail, we explain how Open Science practices are embedded into the so-called 'co-pilot' system in our lab. The 'co-pilot' researcher is involved in all tasks of the 'pilot' researcher, that is designing a study, double-checking experimental and data analysis scripts, as well as writing the manuscript. The lab has set up this co-pilot system to increase transparency, reduce potential errors that could occur during the entire workflow, and to intensify collaborations between lab members. Finally, we discuss potential solutions for general problems that could arise when practicing Open Science.

Correction to 'Open science and modified funding lotteries can impede the natural selection of bad science'

[This corrects the article DOI: 10.1098/rsos.190194.].

CORP: Practical tools for improving experimental design and reporting of laboratory studies of cardiovascular physiology and metabolism

The exercise consisted of: 1) a short survey to acquire baseline data on current practices regarding the conduct of animal studies, 2) a series of presentations for promoting awareness and providing advice and practical tools for improving experimental design, and 3) a follow-up survey 12 mo later to assess whether practices had changed. The surveys were compulsory for responsible investigators (n = 16; paired data presented). Other investigators named on animal ethics applications were encouraged to participate (2017, total of 36 investigators; 2018, 37 investigators). The major findings to come from the exercise included 1) a willingness of investigators to make changes when provided with knowledge/tools and solutions that were relatively simple to implement (e.g., proportion of responsible investigators showing improved practices using a structured method for randomization was 0.44, 95% CI (0.19; 0.70), P = 0.003, and deidentifying drugs/interventions was 0.40, 95% CI (0.12; 0.68), P = 0.010); 2) resistance to change if this involved more personnel and time (e.g., as required for allocation concealment); and 3) evidence that changes to long-term practices ("habits") require time and follow-up. Improved practices could be verified based on changes in reporting within publications or documented evidence provided during laboratory visits. In summary, this exercise resulted in changed attitudes, practices, and reporting, but continued follow-up, monitoring, and incentives are required. Efforts to improve experimental rigor will reduce bias and will lead to findings with the greatest translational potential.NEW & NOTEWORTHY The goal of this exercise was to encourage preclinical researchers to improve the quality of their cardiac and metabolic animal studies by 1) increasing awareness of concerns, which can arise from suboptimal experimental designs; 2) providing knowledge, tools, and templates to overcome bias; and 3) conducting two short surveys over 12 mo to monitor change. Improved practices were identified for the uptake of structured methods for randomization, and de-identifying interventions/drugs.Listen to this article's corresponding podcast at https://ajpheart.podbean.com/e/experimental-design-survey-training-practical-tools/.

Open science and modified funding lotteries can impede the natural selection of bad science

Assessing scientists using exploitable metrics can lead to the degradation of research methods even without any strategic behaviour on the part of individuals, via 'the natural selection of bad science.' Institutional incentives to maximize metrics like publication quantity and impact drive this dynamic. Removing these incentives is necessary, but institutional change is slow. However, recent developments suggest possible solutions with more rapid onsets. These include what we call open science improvements, which can reduce publication bias and improve the efficacy of peer review. In addition, there have been increasing calls for funders to move away from prestige- or innovation-based approaches in favour of lotteries. We investigated whether such changes are likely to improve the reproducibility of science even in the presence of persistent incentives for publication quantity through computational modelling. We found that modified lotteries, which allocate funding randomly among proposals that pass a threshold for methodological rigour, effectively reduce the rate of false discoveries, particularly when paired with open science improvements that increase the publication of negative results and improve the quality of peer review. In the absence of funding that targets rigour, open science improvements can still reduce false discoveries in the published literature but are less likely to improve the overall culture of research practices that underlie those publications.

Open science challenges, benefits and tips in early career and beyond

The movement towards open science is a consequence of seemingly pervasive failures to replicate previous research. This transition comes with great benefits but also significant challenges that are likely to affect those who carry out the research, usually early career researchers (ECRs). Here, we describe key benefits, including reputational gains, increased chances of publication, and a broader increase in the reliability of research. The increased chances of publication are supported by exploratory analyses indicating null findings are substantially more likely to be published via open registered reports in comparison to more conventional methods. These benefits are balanced by challenges that we have encountered and that involve increased costs in terms of flexibility, time, and issues with the current incentive structure, all of which seem to affect ECRs acutely. Although there are major obstacles to the early adoption of open science, overall open science practices should benefit both the ECR and improve the quality of research. We review 3 benefits and 3 challenges and provide suggestions from the perspective of ECRs for moving towards open science practices, which we believe scientists and institutions at all levels would do well to consider.

Presidential address: Six open questions to genetic epidemiologists

Given the rapid pace with which genomics and other ‐omics disciplines are evolving, it is sometimes necessary to shift down a gear to consider more general scientific questions. In this line, in my presidential address I formulate six questions for genetic epidemiologists to ponder on. These cover the areas of reproducibility, statistical significance, chance findings, precision medicine and related fields such as bioinformatics and data science. Possible hints at responses are presented to foster our further discussion of these topics.