A consensus-based transparency checklist

Heterogeneity in effect size estimates

A typical empirical study involves choosing a sample, a research design, and an analysis path. Variation in such choices across studies leads to heterogeneity in results that introduce an additional layer of uncertainty, limiting the generalizability of published scientific findings. We provide a framework for studying heterogeneity in the social sciences and divide heterogeneity into population, design, and analytical heterogeneity. Our framework suggests that after accounting for heterogeneity, the probability that the tested hypothesis is true for the average population, design, and analysis path can be much lower than implied by nominal error rates of statistically significant individual studies. We estimate each type's heterogeneity from 70 multilab replication studies, 11 prospective meta-analyses of studies employing different experimental designs, and 5 multianalyst studies. In our data, population heterogeneity tends to be relatively small, whereas design and analytical heterogeneity are large. Our results should, however, be interpreted cautiously due to the limited number of studies and the large uncertainty in the heterogeneity estimates. We discuss several ways to parse and account for heterogeneity in the context of different methodologies.

Supporting study registration to reduce research waste

An estimated 82-89% of ecological research and 85% of medical research has limited or no value to the end user because of various inefficiencies. We argue that registration and registered reports can enhance the quality and impact of ecological research. Drawing on evidence from other fields, chiefly medicine, we support our claim that registration can reduce research waste. However, increasing registration rates, quality and impact will be very slow without coordinated effort of funders, publishers and research institutions. We therefore call on them to facilitate the adoption of registration by providing adequate support. We outline several aspects to be considered when designing a registration system that would best serve the field of ecology. To further inform the development of such a system, we call for more research to identify the causes of low registration rates in ecology. We suggest short- and long-term actions to bolster registration and reduce research waste.

Subjective evidence evaluation survey for many-analysts studies

Many-analysts studies explore how well an empirical claim withstands plausible alternative analyses of the same dataset by multiple, independent analysis teams. Conclusions from these studies typically rely on a single outcome metric (e.g. effect size) provided by each analysis team. Although informative about the range of plausible effects in a dataset, a single effect size from each team does not provide a complete, nuanced understanding of how analysis choices are related to the outcome. We used the Delphi consensus technique with input from 37 experts to develop an 18-item subjective evidence evaluation survey (SEES) to evaluate how each analysis team views the methodological appropriateness of the research design and the strength of evidence for the hypothesis. We illustrate the usefulness of the SEES in providing richer evidence assessment with pilot data from a previous many-analysts study.

Understanding 'error' in the forensic sciences: A primer

This paper distils seven key lessons about 'error' from a collaborative webinar series between practitioners at Victoria Police Forensic Services Department and academics. It aims to provide the common understanding of error necessary to foster interdisciplinary dialogue, collaboration and research. The lessons underscore the inevitability, complexity and subjectivity of error, as well as opportunities for learning and growth. Ultimately, we argue that error can be a potent tool for continuous improvement and accountability, enhancing the reliability of forensic sciences and public trust. It is hoped the shared understanding provided by this paper will support future initiatives and funding for collaborative developments in this vital domain.

Establishing circularity: development and validation of the circular work value scale (CWVS)

Objectives

Addressing the lack of German-language instruments, this study aims to develop a questionnaire that enables the measurement of work values. According to the theory of basic human values (Schwartz, 1992), a culturally fitting questionnaire is validated by covering constructs in the four broader dimensions of Social, Prestige, Intrinsic, and Extrinsic work values. Convergent, discriminant and incremental congruent validity are assessed.

Method

Data were collected in a cross-sectional online-based panel survey. Individuals working more than 20 h per week were included (N = 1,049). Using a genetic algorithm, an economical and valid questionnaire was designed to assess work values.

Results

The 11 work values are measurable with three items each. They provide a good fit to the data with support for strict measurement invariance. The empirical associations to estimate construct validity overall reflect expected relations to social and individualistic work motives, neuroticism, environmental awareness, and basic values. Furthermore, congruent incremental validity is supported with relations to value congruence of the person-organization fit, and multidimensional scaling supports the assumed theoretical circularity of the work values.

Implications

This study developed a questionnaire that enables a theory-based valid measurement of work values. The questionnaire allows practitioners to economically collect information about the value structure of employees or applicants. Future research should consider the development of work values over time and investigate whether more distinctive constructs provide a better fit in the nomological network.

Ten Ways to Improve Getting a Scientific Manuscript Accepted

BACKGROUND

Scientific publication is the cornerstone to academic and private practice advancement in patient management and outcomes. Writing a manuscript requires a certain discipline and skill set that can be achieved with diligence and hard work.

METHODS

Anecdotal and review.

RESULTS

Several factors must be considered in scientific writing and journal manuscript submission and acceptance. Choosing where to submit the manuscript; understanding the instructions to authors; disclosing ethically; formatting correctly; never plagiarizing; supplying high quality appropriate images; creating meaningful tables; curating a pertinent but thorough bibliography; having valid, supported conclusions; and respecting timelines.

CONCLUSION

A discussion of relevant components in manuscript writing and journal submission to improve your chances of acceptance.

Fifty years of research on questionable research practises in science: quantitative analysis of co-citation patterns

Questionable research practises (QRPs) have been the focus of the scientific community amid greater scrutiny and evidence highlighting issues with replicability across many fields of science. To capture the most impactful publications and the main thematic domains in the literature on QRPs, this study uses a document co-citation analysis. The analysis was conducted on a sample of 341 documents that covered the past 50 years of research in QRPs. Nine major thematic clusters emerged. Statistical reporting and statistical power emerged as key areas of research, where systemic-level factors in how research is conducted are consistently raised as the precipitating factors for QRPs. There is also an encouraging shift in the focus of research into open science practises designed to address engagement in QRPs. Such a shift is indicative of the growing momentum of the open science movement, and more research can be conducted on how these practises are employed on the ground and how their uptake by researchers can be further promoted. However, the results suggest that, while pre-registration and registered reports receive the most research interest, less attention has been paid to other open science practises (e.g. data sharing).

Disclosure Standards for Social Media and Generative Artificial Intelligence Research: Toward Transparency and Replicability

Social media dominate today's information ecosystem and provide valuable information for social research. Market researchers, social scientists, policymakers, government entities, public health researchers, and practitioners recognize the potential for social data to inspire innovation, support products and services, characterize public opinion, and guide decisions. The appeal of mining these rich datasets is clear. However, there is potential risk of data misuse, underscoring an equally huge and fundamental flaw in the research: there are no procedural standards and little transparency. Transparency across the processes of collecting and analyzing social media data is often limited due to proprietary algorithms. Spurious findings and biases introduced by artificial intelligence (AI) demonstrate the challenges this lack of transparency poses for research. Social media research remains a virtual "wild west," with no clear standards for reporting regarding data retrieval, preprocessing steps, analytic methods, or interpretation. Use of emerging generative AI technologies to augment social media analytics can undermine validity and replicability of findings, potentially turning this research into a "black box" enterprise. Clear guidance for social media analyses and reporting is needed to assure the quality of the resulting research. In this article, we propose criteria for evaluating the quality of studies using social media data, grounded in established scientific practice. We offer clear documentation guidelines to ensure that social data are used properly and transparently in research and applications. A checklist of disclosure elements to meet minimal reporting standards is proposed. These criteria will make it possible for scholars and practitioners to assess the quality, credibility, and comparability of research findings using digital data.

Acceptability of Personal Sensing Among People With Alcohol Use Disorder: Observational Study

BACKGROUND

Personal sensing may improve digital therapeutics for mental health care by facilitating early screening, symptom monitoring, risk prediction, and personalized adaptive interventions. However, further development and the use of personal sensing requires a better understanding of its acceptability to people targeted for these applications.

OBJECTIVE

We aimed to assess the acceptability of active and passive personal sensing methods in a sample of people with moderate to severe alcohol use disorder using both behavioral and self-report measures. This sample was recruited as part of a larger grant-funded project to develop a machine learning algorithm to predict lapses.

METHODS

Participants (N=154; n=77, 50% female; mean age 41, SD 11.9 years; n=134, 87% White and n=150, 97% non-Hispanic) in early recovery (1-8 weeks of abstinence) were recruited to participate in a 3-month longitudinal study. Participants were modestly compensated for engaging with active (eg, ecological momentary assessment [EMA], audio check-in, and sleep quality) and passive (eg, geolocation, cellular communication logs, and SMS text message content) sensing methods that were selected to tap into constructs from the Relapse Prevention model by Marlatt. We assessed 3 behavioral indicators of acceptability: participants' choices about their participation in the study at various stages in the procedure, their choice to opt in to provide data for each sensing method, and their adherence to a subset of the active methods (EMA and audio check-in). We also assessed 3 self-report measures of acceptability (interference, dislike, and willingness to use for 1 year) for each method.

RESULTS

Of the 192 eligible individuals screened, 191 consented to personal sensing. Most of these individuals (169/191, 88.5%) also returned 1 week later to formally enroll, and 154 participated through the first month follow-up visit. All participants in our analysis sample opted in to provide data for EMA, sleep quality, geolocation, and cellular communication logs. Out of 154 participants, 1 (0.6%) did not provide SMS text message content and 3 (1.9%) did not provide any audio check-ins. The average adherence rate for the 4 times daily EMA was .80. The adherence rate for the daily audio check-in was .54. Aggregate participant ratings indicated that all personal sensing methods were significantly more acceptable (all P<.001) compared with neutral across subjective measures of interference, dislike, and willingness to use for 1 year. Participants did not significantly differ in their dislike of active methods compared with passive methods (P=.23). However, participants reported a higher willingness to use passive (vs active) methods for 1 year (P=.04).

CONCLUSIONS

These results suggest that active and passive sensing methods are acceptable for people with alcohol use disorder over a longer period than has previously been assessed. Important individual differences were observed across people and methods, indicating opportunities for future improvement.

Diagnostic Approach and Differences between Spinal Infections and Tumors

STUDY DESIGN

A systematic review of the literature about differential diagnosis between spine infection and bone tumors of the spine.

BACKGROUND AND PURPOSE

The differential diagnosis between spine infection and bone tumors of the spine can be misled by the prevalence of one of the conditions over the other in different areas of the world. A review of the existing literature on suggestive or even pathognomonic imaging aspects of both can be very useful for correctly orientating the diagnosis and deciding the most appropriate area for biopsy. The purpose of our study is to identify which imaging technique is the most reliable to suggest the diagnosis between spine infection and spine bone tumor.

METHODS

A primary search on Medline through PubMed distribution was made. We identified five main groups: tuberculous, atypical spinal tuberculosis, pyogenic spondylitis, and neoplastic (primitive and metastatic). For each group, we evaluated the commonest localization, characteristics at CT, CT perfusion, MRI, MRI with Gadolinium, MRI diffusion (DWI) and, in the end, the main features for each group.

RESULTS

A total of 602 studies were identified through the database search and a screening by titles and abstracts was performed. After applying inclusion and exclusion criteria, 34 articles were excluded and a total of 22 full-text articles were assessed for eligibility. For each article, the role of CT-scan, CT-perfusion, MRI, MRI with Gadolinium and MRI diffusion (DWI) in distinguishing the most reliable features to suggest the diagnosis of spine infection versus bone tumor/metastasis was collected.

CONCLUSION

Definitive differential diagnosis between infection and tumor requires biopsy and culture. The sensitivity and specificity of percutaneous biopsy are 72% and 94%, respectively. Imaging studies can be added to address the diagnosis, but a multidisciplinary discussion with radiologists and nuclear medicine specialists is mandatory.

Combating COVID-19 with charisma: Evidence on governor speeches in the United States

Using field and laboratory data, we show that leader charisma can affect COVID-related mitigating behaviors. We coded a panel of U.S. governor speeches for charisma signaling using a deep neural network algorithm. The model explains variation in stay-at-home behavior of citizens based on their smart phone data movements, showing a robust effect of charisma signaling: stay-at-home behavior increased irrespective of state-level citizen political ideology or governor party allegiance. Republican governors with a particularly high charisma signaling score impacted the outcome more relative to Democratic governors in comparable conditions. Our results also suggest that one standard deviation higher charisma signaling in governor speeches could potentially have saved 5,350 lives during the study period (02/28/2020-05/14/2020). Next, in an incentivized laboratory experiment we found that politically conservative individuals are particularly prone to believe that their co-citizens will follow governor appeals to distance or stay at home when exposed to a speech that is high in charisma; these beliefs in turn drive their preference to engage in those behaviors. These results suggest that political leaders should consider additional "soft-power" levers like charisma-which can be learned-to complement policy interventions for pandemics or other public heath crises, especially with certain populations who may need a "nudge."

A direct replication and extension of Popp and Serra (2016, experiment 1): better free recall and worse cued recall of animal names than object names, accounting for semantic similarity

Introduction

Free recall tends to be better for names of animate concepts such as animals than for names of inanimate objects. In Popp and Serra's 2016 article, the authors replicated this "animacy effect" in free recall but when participants studied words in pairs (animate-animate pairs intermixed with inanimate-inanimate pairs) and were tested with cued recall, performance was better for inanimate-inanimate pairs than for animate-animate pairs ("reverse animacy"). We tested the replicability of this surprising effect and one possible explanation for the effect (semantic similarity).

Methods

Our Experiment 1 was a preregistered direct replication (N = 101) of Popp and Serra's Experiment 1 (mixed-lists condition). In a second preregistered experiment conducted in four different samples (undergraduate N = 153, undergraduate N = 143, online Prolific N = 101, online Prolific/English-as-a-first-language N = 150), we manipulated the within-category semantic similarity of animal and object wordlists.

Results

AIn Experiment 1, just as in Popp and Serra, we observed an animacy effect for free recall and a reverse animacy effect for cued recall. Unlike Popp and Serra, we found that controlling for interference effects rendered the reverse animacy effect non-significant. We took this as evidence that characteristics of the stimulus sets (e.g., category structure, within-category similarity) may play a role in animacy and reverse animacy effects. In Experiment 2, in three out of our four samples, we observed reverse animacy effects when within-category similarity was higher for animals and when within-category similarity was equated for animals and objects.

Discussion

Our results suggest that the reverse animacy effect observed in Popp and Serra's 2016 article is a robust and replicable effect, but that semantic similarity alone cannot explain the effect.

Using preregistration as a tool for transparent fNIRS study design

Significance

The expansion of functional near-infrared spectroscopy (fNIRS) methodology and analysis tools gives rise to various design and analytical decisions that researchers have to make. Several recent efforts have developed guidelines for preprocessing, analyzing, and reporting practices. For the planning stage of fNIRS studies, similar guidance is desirable. Study preregistration helps researchers to transparently document study protocols before conducting the study, including materials, methods, and analyses, and thus, others to verify, understand, and reproduce a study. Preregistration can thus serve as a useful tool for transparent, careful, and comprehensive fNIRS study design.

Aim

We aim to create a guide on the design and analysis steps involved in fNIRS studies and to provide a preregistration template specified for fNIRS studies.

Approach

The presented preregistration guide has a strong focus on fNIRS specific requirements, and the associated template provides examples based on continuous-wave (CW) fNIRS studies conducted in humans. These can, however, be extended to other types of fNIRS studies.

Results

On a step-by-step basis, we walk the fNIRS user through key methodological and analysis-related aspects central to a comprehensive fNIRS study design. These include items specific to the design of CW, task-based fNIRS studies, but also sections that are of general importance, including an in-depth elaboration on sample size planning.

Conclusions

Our guide introduces these open science tools to the fNIRS community, providing researchers with an overview of key design aspects and specification recommendations for comprehensive study planning. As such it can be used as a template to preregister fNIRS studies or merely as a tool for transparent fNIRS study design.

Raising the value of research studies in psychological science by increasing the credibility of research reports: the transparent Psi project

The low reproducibility rate in social sciences has produced hesitation among researchers in accepting published findings at their face value. Despite the advent of initiatives to increase transparency in research reporting, the field is still lacking tools to verify the credibility of research reports. In the present paper, we describe methodologies that let researchers craft highly credible research and allow their peers to verify this credibility. We demonstrate the application of these methods in a multi-laboratory replication of Bem's Experiment 1 (Bem 2011 J. Pers. Soc. Psychol. 100, 407-425. (doi:10.1037/a0021524)) on extrasensory perception (ESP), which was co-designed by a consensus panel including both proponents and opponents of Bem's original hypothesis. In the study we applied direct data deposition in combination with born-open data and real-time research reports to extend transparency to protocol delivery and data collection. We also used piloting, checklists, laboratory logs and video-documented trial sessions to ascertain as-intended protocol delivery, and external research auditors to monitor research integrity. We found 49.89% successful guesses, while Bem reported 53.07% success rate, with the chance level being 50%. Thus, Bem's findings were not replicated in our study. In the paper, we discuss the implementation, feasibility and perceived usefulness of the credibility-enhancing methodologies used throughout the project.

The process of replication target selection in psychology: what to consider?

Increased execution of replication studies contributes to the effort to restore credibility of empirical research. However, a second generation of problems arises: the number of potential replication targets is at a serious mismatch with available resources. Given limited resources, replication target selection should be well-justified, systematic and transparently communicated. At present the discussion on what to consider when selecting a replication target is limited to theoretical discussion, self-reported justifications and a few formalized suggestions. In this Registered Report, we proposed a study involving the scientific community to create a list of considerations for consultation when selecting a replication target in psychology. We employed a modified Delphi approach. First, we constructed a preliminary list of considerations. Second, we surveyed psychologists who previously selected a replication target with regards to their considerations. Third, we incorporated the results into the preliminary list of considerations and sent the updated list to a group of individuals knowledgeable about concerns regarding replication target selection. Over the course of several rounds, we established consensus regarding what to consider when selecting a replication target. The resulting checklist can be used for transparently communicating the rationale for selecting studies for replication.

[The PRISMA 2020 statement: an updated guideline for reporting systematic reviewsDeclaración PRISMA 2020: una guía actualizada para la publicación de revisiones sistemáticas]

The Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement, published in 2009, was designed to help systematic reviewers transparently report why the review was done, what the authors did, and what they found. Over the past decade, advances in systematic review methodology and terminology have necessitated an update to the guideline. The PRISMA 2020 statement replaces the 2009 statement and includes new reporting guidance that reflects advances in methods to identify, select, appraise, and synthesise studies. The structure and presentation of the items have been modified to facilitate implementation. In this article, we present the PRISMA 2020 27-item checklist, an expanded checklist that details reporting recommendations for each item, the PRISMA 2020 abstract checklist, and the revised flow diagrams for original and updated reviews.

A critique of using the labels confirmatory and exploratory in modern psychological research

Psychological science is experiencing a rise in the application of complex statistical models and, simultaneously, a renewed focus on applying research in a confirmatory manner. This presents a fundamental conflict for psychological researchers as more complex forms of modeling necessarily eschew as stringent of theoretical constraints. In this paper, I argue that this is less of a conflict, and more a result of a continued adherence to applying the overly simplistic labels of exploratory and confirmatory. These terms mask a distinction between exploratory/confirmatory research practices and modeling. Further, while many researchers recognize that this dichotomous distinction is better represented as a continuum, this only creates additional problems. Finally, I argue that while a focus on preregistration helps clarify the distinction, psychological research would be better off replacing the terms exploratory and confirmatory with additional levels of detail regarding the goals of the study, modeling details, and scientific method.

A Bayesian Modeling Approach to Examine the Role of Testosterone Administration on the Endowment Effect and Risk-Taking

Financial risk-taking and loss aversion are multifaceted phenomena that are the focus of neuroscience, psychology, and economics research. A growing number of studies highlighted the role of hormones (particularly of testosterone) on socio-economic decision-making. However, the effects of testosterone on risk-taking under framing and consumer-based choices and preferences are inconclusive. We investigated the effects of 100 mg testosterone administration on aspects of decision-making within the Prospect Theory framework which is the most used descriptive model of decision-making under risk. We assessed risk-taking under framing and the endowment effect (effect of possession) using Bayesian modeling. Forty men participated in this double-blind placebo-controlled fully-randomized cross-over experiment and performed two tasks. One was a risk-taking task with binary choices under positive and negative framing associated with different probabilities. In the second task participants had to bid money for hedonic and utilitarian items. We observed a significant increase in serum testosterone concentrations after transdermal application. Compared to placebo, testosterone administration increased risk-taking under the positive framing (very large effect size) and decreased under the negative framing (moderate to small). The sensitivity to gain was positive in each framing. Our model showed that decision-making is jointly influenced by testosterone and the trade-off between gains and losses. However, while the endowment effect was more pronounced for hedonic than for utilitarian items, the effect was independent of testosterone. The findings provide novel information on the complex modulatory role of testosterone on risk-taking within the framework of prospect theory and shed light on mechanisms of behavioral economic biases. The proposed models of effects of individual differences in testosterone on risk-taking could be used as predictive models for reference-depended behavior under positive and negative framing with low and high probabilities.

A survey on how preregistration affects the research workflow: better science but more work

The preregistration of research protocols and analysis plans is a main reform innovation to counteract confirmation bias in the social and behavioural sciences. While theoretical reasons to preregister are frequently discussed in the literature, the individually experienced advantages and disadvantages of this method remain largely unexplored. The goal of this exploratory study was to identify the perceived benefits and challenges of preregistration from the researcher's perspective. To this end, we surveyed 355 researchers, 299 of whom had used preregistration in their own work. The researchers indicated the experienced or expected effects of preregistration on their workflow. The results show that experiences and expectations are mostly positive. Researchers in our sample believe that implementing preregistration improves or is likely to improve the quality of their projects. Criticism of preregistration is primarily related to the increase in work-related stress and the overall duration of the project. While the benefits outweighed the challenges for the majority of researchers with preregistration experience, this was not the case for the majority of researchers without preregistration experience. The experienced advantages and disadvantages identified in our survey could inform future efforts to improve preregistration and thus help the methodology gain greater acceptance in the scientific community.

What senior academics can do to support reproducible and open research: a short, three-step guide

Increasingly, policies are being introduced to reward and recognise open research practices, while the adoption of such practices into research routines is being facilitated by many grassroots initiatives. However, despite this widespread endorsement and support, as well as various efforts led by early career researchers, open research is yet to be widely adopted. For open research to become the norm, initiatives should engage academics from all career stages, particularly senior academics (namely senior lecturers, readers, professors) given their routine involvement in determining the quality of research. Senior academics, however, face unique challenges in implementing policy changes and supporting grassroots initiatives. Given that-like all researchers-senior academics are motivated by self-interest, this paper lays out three feasible steps that senior academics can take to improve the quality and productivity of their research, that also serve to engender open research. These steps include changing (a) hiring criteria, (b) how scholarly outputs are credited, and (c) how we fund and publish in line with open research principles. The guidance we provide is accompanied by material for further reading.

Combine Statistical Thinking With Open Scientific Practice: A Protocol of a Bayesian Research Project

Current developments in the statistics community suggest that modern statistics education should be structured holistically, that is, by allowing students to work with real data and to answer concrete statistical questions, but also by educating them about alternative frameworks, such as Bayesian inference. In this article, we describe how we incorporated such a holistic structure in a Bayesian research project on ordered binomial probabilities. The project was conducted with a group of three undergraduate psychology students who had basic knowledge of Bayesian statistics and programming, but lacked formal mathematical training. The research project aimed to (1) convey the basic mathematical concepts of Bayesian inference; (2) have students experience the entire empirical cycle including collection, analysis, and interpretation of data and (3) teach students open science practices.

Unpalatable truths: Commitment to eating meat is associated with strategic ignorance of food-animal minds

Animal minds are of central importance to debates about their rights and welfare. Remaining ignorant of evidence that animals have minds is therefore likely to facilitate their mistreatment. Studying samples of adults and students from the UK and US we found that, consistent with motivational perspectives on meat consumption, those who were more (vs. less) committed to eating meat were more motivated to avoid exposure to information about food-animals' sentience (Studies 1), showed less interest in exposure to articles about intelligent food animals (Studies 2a and 2b), and were quicker to terminate exposure to internet pop-ups containing information about food-animals' minds (Studies 3a and 3b). At the same time, those who were more (vs. less) committed to eating meat approached information about companion-animals' minds (Studies 2a-3b) and unintelligent food animals (Studies 2a and 2b) in largely the same ways. The findings demonstrate that, within the UK and US, the desire to eat meat is associated with strategies to avoid information that is likely to challenge meat consumption.

A machine learning analysis of the relationship of demographics and social gathering attendance from 41 countries during pandemic

Knowing who to target with certain messages is the prerequisite of efficient public health campaigns during pandemics. Using the COVID-19 pandemic situation, we explored which facets of the society-defined by age, gender, income, and education levels-are the most likely to visit social gatherings and aggravate the spread of a disease. Analyzing the reported behavior of 87,169 individuals from 41 countries, we found that in the majority of the countries, the proportion of social gathering-goers was higher in male than female, younger than older, lower-educated than higher educated, and low-income than high-income subgroups of the populations. However, the data showed noteworthy heterogeneity between the countries warranting against generalizing from one country to another. The analysis also revealed that relative to other demographic factors, income was the strongest predictor of avoidance of social gatherings followed by age, education, and gender. Although the observed strength of these associations was relatively small, we argue that incorporating demographic-based segmentation into public health campaigns can increase the efficiency of campaigns with an important caveat: the exploration of these associations needs to be done on a country level before using the information to target populations in behavior change interventions.

Registered report: How open do you want your science? An international investigation into knowledge and attitudes of psychology students

The use of Open Science practices is often proposed as a way to improve research practice, especially in psychology. Open Science can increase transparency and therefore reduce questionable research practices, making research more accessible to students, scholars, policy makers, and the public. However, little is known about how widespread Open Science practices are taught and how students are educated about these practices. In addition, it remains unknown how informing students about Open Science actually impacts their understanding and adoption of such practices. This registered report proposes the validation of a questionnaire. The aim is to survey how much psychology students know about Open Science and to assess whether knowledge of and exposure to Open Science in general-be it through university curricula or social media-influences attitudes towards the concept and intentions to implement relevant practices.

Developing an open science 'mindset'

BACKGROUND

Identification of widespread biases present in reported research findings in many scientific disciplines, including psychology, such as failures to replicate and the likely extensive application of questionable research practices, has raised serious concerns over the reliability and trustworthiness of scientific research. This has led to the development of, and advocacy for, 'open science' practices, including data, materials, analysis, and output sharing, pre-registration of study predictions and analysis plans, and increased access to published research findings. Implementation of such practices has been enthusiastic in some quarters, but literacy in, and adoption of, these practices has lagged behind among many researchers in the scientific community.

ADVANCES

In the current article I propose that researchers adopt an open science 'mindset', a comprehensive approach to open science predicated on researchers' operating under the basic assumption that, wherever possible, open science practices will be a central component of all steps of their research projects. The primary, defining feature of the mindset is a commitment to open science principles in all research projects from inception to dissemination. Other features of the mindset include the assumption that all components of research projects (e.g. pre-registered hypotheses, protocols, materials, analysis plans, data, and output) will be accessible broadly; pro-active selection of open fora to disseminate research components and findings; open and transparent dissemination of reports of the research findings in advance of, and after, formal publication; and active promotion of open science practices through education, modeling, and advocacy.

CONCLUSION

The open science mindset is a 'farm to fork' approach to open science aimed at promoting comprehensive quality in application of open science, and widening participation in open science practices so that they become the norm in research in health psychology and behavioral medicine going forward.

Consensus-based guidance for conducting and reporting multi-analyst studies

Any large dataset can be analyzed in a number of ways, and it is possible that the use of different analysis strategies will lead to different results and conclusions. One way to assess whether the results obtained depend on the analysis strategy chosen is to employ multiple analysts and leave each of them free to follow their own approach. Here, we present consensus-based guidance for conducting and reporting such multi-analyst studies, and we discuss how broader adoption of the multi-analyst approach has the potential to strengthen the robustness of results and conclusions obtained from analyses of datasets in basic and applied research.

Seven steps toward more transparency in statistical practice

We argue that statistical practice in the social and behavioural sciences benefits from transparency, a fair acknowledgement of uncertainty and openness to alternative interpretations. Here, to promote such a practice, we recommend seven concrete statistical procedures: (1) visualizing data; (2) quantifying inferential uncertainty; (3) assessing data preprocessing choices; (4) reporting multiple models; (5) involving multiple analysts; (6) interpreting results modestly; and (7) sharing data and code. We discuss their benefits and limitations, and provide guidelines for adoption. Each of the seven procedures finds inspiration in Merton's ethos of science as reflected in the norms of communalism, universalism, disinterestedness and organized scepticism. We believe that these ethical considerations-as well as their statistical consequences-establish common ground among data analysts, despite continuing disagreements about the foundations of statistical inference.

Publication bias and low power in field studies on goal priming

Research on goal priming asks whether the subtle activation of an achievement goal can improve task performance. Studies in this domain employ a range of priming methods, such as surreptitiously displaying a photograph of an athlete winning a race, and a range of dependent variables including measures of creativity and workplace performance. Chen, Latham, Piccolo and Itzchakov (Chen et al. 2021 J. Appl. Psychol. 70, 216-253) recently undertook a meta-analysis of this research and reported positive overall effects in both laboratory and field studies, with field studies yielding a moderate-to-large effect that was significantly larger than that obtained in laboratory experiments. We highlight a number of issues with Chen et al.'s selection of field studies and then report a new meta-analysis (k = 13, N = 683) that corrects these. The new meta-analysis reveals suggestive evidence of publication bias and low power in goal priming field studies. We conclude that the available evidence falls short of demonstrating goal priming effects in the workplace, and offer proposals for how future research can provide stronger tests.

Preferred reporting items for systematic reviews and meta-analyses in ecology and evolutionary biology: a PRISMA extension

Since the early 1990s, ecologists and evolutionary biologists have aggregated primary research using meta-analytic methods to understand ecological and evolutionary phenomena. Meta-analyses can resolve long-standing disputes, dispel spurious claims, and generate new research questions. At their worst, however, meta-analysis publications are wolves in sheep's clothing: subjective with biased conclusions, hidden under coats of objective authority. Conclusions can be rendered unreliable by inappropriate statistical methods, problems with the methods used to select primary research, or problems within the primary research itself. Because of these risks, meta-analyses are increasingly conducted as part of systematic reviews, which use structured, transparent, and reproducible methods to collate and summarise evidence. For readers to determine whether the conclusions from a systematic review or meta-analysis should be trusted - and to be able to build upon the review - authors need to report what they did, why they did it, and what they found. Complete, transparent, and reproducible reporting is measured by 'reporting quality'. To assess perceptions and standards of reporting quality of systematic reviews and meta-analyses published in ecology and evolutionary biology, we surveyed 208 researchers with relevant experience (as authors, reviewers, or editors), and conducted detailed evaluations of 102 systematic review and meta-analysis papers published between 2010 and 2019. Reporting quality was far below optimal and approximately normally distributed. Measured reporting quality was lower than what the community perceived, particularly for the systematic review methods required to measure trustworthiness. The minority of assessed papers that referenced a guideline (~16%) showed substantially higher reporting quality than average, and surveyed researchers showed interest in using a reporting guideline to improve reporting quality. The leading guideline for improving reporting quality of systematic reviews is the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement. Here we unveil an extension of PRISMA to serve the meta-analysis community in ecology and evolutionary biology: PRISMA-EcoEvo (version 1.0). PRISMA-EcoEvo is a checklist of 27 main items that, when applicable, should be reported in systematic review and meta-analysis publications summarising primary research in ecology and evolutionary biology. In this explanation and elaboration document, we provide guidance for authors, reviewers, and editors, with explanations for each item on the checklist, including supplementary examples from published papers. Authors can consult this PRISMA-EcoEvo guideline both in the planning and writing stages of a systematic review and meta-analysis, to increase reporting quality of submitted manuscripts. Reviewers and editors can use the checklist to assess reporting quality in the manuscripts they review. Overall, PRISMA-EcoEvo is a resource for the ecology and evolutionary biology community to facilitate transparent and comprehensively reported systematic reviews and meta-analyses.

Objectivity for the research worker

In the last decade, many problematic cases of scientific conduct have been diagnosed; some of which involve outright fraud (e.g., Stapel, 2012) others are more subtle (e.g., supposed evidence of extrasensory perception; Bem, 2011). These and similar problems can be interpreted as caused by lack of scientific objectivity. The current philosophical theories of objectivity do not provide scientists with conceptualizations that can be effectively put into practice in remedying these issues. We propose a novel way of thinking about objectivity for individual scientists; a negative and dynamic approach.We provide a philosophical conceptualization of objectivity that is informed by empirical research. In particular, it is our intention to take the first steps in providing an empirically and methodologically informed inventory of factors that impair the scientific practice. The inventory will be compiled into a negative conceptualization (i.e., what is not objective), which could in principle be used by individual scientists to assess (deviations from) objectivity of scientific practice. We propose a preliminary outline of a usable and testable instrument for indicating the objectivity of scientific practice.

Development of a Questionnaire Measuring Preventive Behaviors at Work

Purpose This study aimed to develop a questionnaire measuring preventive behaviors at work. Methods A three-step design, including qualitative and quantitative methods, was followed: (1) item generation, (2) experts' validation of content, and (3) pretesting. Results For step 1, 49 relevant existing scales were reviewed, and a pool of 172 items was generated. Redundant items were deleted (n = 48), and unclear items were reworded (n = 27). For step 2, 14 experts (five occupational therapists, four researchers, and five workers) assessed the representativeness, relevance, and clarity of each item through content validity indices (CVIs). An average overall CVI of 0.97 was obtained, and 87.5% of the experts stated that the questionnaire was comprehensive. During this step, 63 items were deleted, and 35 were modified. For step 3, the tool was pretested in the clinical settings of four dyads (occupational therapist-worker). The thematic analysis of interview content allowed several changes to be made to the questionnaire, including the addition of information and format changes. Conclusions Overall, this three-step study led to the construction of a 61-item French questionnaire entitled the Échelle de fréquence des comportements préventifs au travail [Frequency Scale of Preventive Behaviors at Work]. In rehabilitation settings, this tool could be useful to support professionals in enabling workers to adopt preventive behaviors, thereby fostering a healthy, sustainable return to work after a disability period. However, further metrological property assessment is required. A validating study using a large pool of workers is ongoing.

The PRISMA 2020 statement: an updated guideline for reporting systematic reviews

The Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement, published in 2009, was designed to help systematic reviewers transparently report why the review was done, what the authors did, and what they found. Over the past decade, advances in systematic review methodology and terminology have necessitated an update to the guideline. The PRISMA 2020 statement replaces the 2009 statement and includes new reporting guidance that reflects advances in methods to identify, select, appraise, and synthesise studies. The structure and presentation of the items have been modified to facilitate implementation. In this article, we present the PRISMA 2020 27-item checklist, an expanded checklist that details reporting recommendations for each item, the PRISMA 2020 abstract checklist, and the revised flow diagrams for original and updated reviews. Full English text available from:www.revespcardiol.org/en.

On the difficulty to think in ratios: a methodological bias in Stevens' magnitude estimation procedure

In the field of new psychophysics, the magnitude estimation procedure is one of the most frequently used methods. It requires participants to assess the intensity of a stimulus in relation to a reference. In three studies, we examined whether difficulties of thinking in ratios influence participants' intensity perceptions. In Study 1, a standard magnitude estimation procedure was compared to an adapted procedure in which the numerical response dimension was reversed so that smaller (larger) numbers indicated brighter (darker) stimuli. In Study 2, participants first had to indicate whether a stimulus was brighter or darker compared to the reference, and only afterwards they estimated the magnitude of this difference, always using ratings above the reference to indicate their perception. In Study 3, we applied the same procedure as in Study 2 to a different physical dimension (red saturation). Results from Study 1 (N = 20) showed that participants in the reversal condition used more (less) extreme ratings for brighter (darker) stimuli compared to the standard condition. Data from the unidirectional method applied in Study 2 (N = 34) suggested a linear psychophysical function for brightness perception. Similar results were found for red saturation in Study 3 (N = 36) with a less curved power function describing the association between objective red saturation and perceived redness perception. We conclude that the typical power functions that emerge when using a standard magnitude estimation procedure might be biased due to difficulties experienced by participants to think in ratios.

Integrating explanation and prediction in computational social science

Computational social science is more than just large repositories of digital data and the computational methods needed to construct and analyse them. It also represents a convergence of different fields with different ways of thinking about and doing science. The goal of this Perspective is to provide some clarity around how these approaches differ from one another and to propose how they might be productively integrated. Towards this end we make two contributions. The first is a schema for thinking about research activities along two dimensions-the extent to which work is explanatory, focusing on identifying and estimating causal effects, and the degree of consideration given to testing predictions of outcomes-and how these two priorities can complement, rather than compete with, one another. Our second contribution is to advocate that computational social scientists devote more attention to combining prediction and explanation, which we call integrative modelling, and to outline some practical suggestions for realizing this goal.

The PRISMA 2020 statement: An updated guideline for reporting systematic reviews

The Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement, published in 2009, was designed to help systematic reviewers transparently report why the review was done, what the authors did, and what they found. Over the past decade, advances in systematic review methodology and terminology have necessitated an update to the guideline. The PRISMA 2020 statement replaces the 2009 statement and includes new reporting guidance that reflects advances in methods to identify, select, appraise, and synthesise studies. The structure and presentation of the items have been modified to facilitate implementation. In this article, we present the PRISMA 2020 27-item checklist, an expanded checklist that details reporting recommendations for each item, the PRISMA 2020 abstract checklist, and the revised flow diagrams for original and updated reviews.