Commentary: Perverse incentives or rotten apples?

Investigating the links between questionable research practices, scientific norms and organisational culture

BACKGROUND

This study investigates the determinants of engagement in questionable research practices (QRPs), focusing on both individual-level factors (such as scholarly field, commitment to scientific norms, gender, contract type, and career stage) and institution-level factors (including industry type, researchers' perceptions of their research culture, and awareness of institutional policies on research integrity).

METHODS

Using a multi-level modelling approach, we analyse data from an international survey of researchers working across disciplinary fields to estimate the effect of these factors on QRP engagement.

RESULTS

Our findings indicate that contract type, career stage, academic field, adherence to scientific norms and gender significantly predict QRP engagement. At the institution level, factors such as being outside of a collegial culture and experiencing harmful publication pressure, and the presence of safeguards against integrity breaches have small associations. Only a minimal amount of variance in QRP engagement is attributable to differences between institutions and countries.

CONCLUSIONS

We discuss the implications of these findings for developing effective interventions to reduce QRPs, highlighting the importance of addressing both individual and institutional factors in efforts to foster research integrity.

Is something rotten in the state of Denmark? Cross-national evidence for widespread involvement but not systematic use of questionable research practices across all fields of research

Questionable research practices (QRP) are believed to be widespread, but empirical assessments are generally restricted to a few types of practices. Furthermore, conceptual confusion is rife with use and prevalence of QRPs often being confused as the same quantity. We present the hitherto most comprehensive study examining QRPs across scholarly fields and knowledge production modes. We survey perception, use, prevalence and predictors of QRPs among 3,402 researchers in Denmark and 1,307 in the UK, USA, Croatia and Austria. Results reveal remarkably similar response patterns among Danish and international respondents (τ = 0.85). Self-reported use indicates whether respondents have used a QRP in recent publications. 9 out of 10 respondents admitted using at least one QRP. Median use is three out of nine QRP items. Self-reported prevalence reflects the frequency of use. On average, prevalence rates were roughly three times lower compared to self-reported use. Findings indicated that the perceived social acceptability of QRPs influenced self-report patterns. Results suggest that most researchers use different types of QRPs within a restricted time period. The prevalence estimates, however, do not suggest outright systematic use of specific QRPs. Perceived pressure was the strongest systemic predictor for prevalence. Conversely, more local attention to research cultures and academic age was negatively related to prevalence. Finally, the personality traits conscientiousness and, to a lesser degree, agreeableness were also inversely associated with self-reported prevalence. Findings suggest that explanations for engagement with QRPs are not only attributable to systemic factors, as hitherto suggested, but a complicated mixture of experience, systemic and individual factors, and motivated reasoning.

The punishment intensity for research misconduct and its related factors: An exploratory study on hospitals in Mainland China

Previous studies have found that factors such as gender and academic positions do not influence the severity of administrative actions taken by institutions. However, this study provides partly inconsistent evidence. It focuses on incidents of research misconduct in hospitals across Mainland China and explores factors related to punishment using a large cross-sectional dataset (N = 815). Regression analysis revealed a significant correlation between authorship order and the punishment intensity (p < 0.05). Under specific conditions, there was a significant correlation between the professional title (senior) and punishment intensity (p = 0.001), and an interaction between professional title and types of research misbehavior. Further analysis of simple effects showed that, in cases of fabrication and falsification, and combinations of multiple research misbehavior, researchers with senior titles received significantly lighter punishments compared to those with junior, intermediate, and associate senior titles (p < 0.05). The study unveils the potential accountability patterns (collective accountability and tiered punishment) that may be adopted by hospitals in Mainland China, as well as the challenges faced in ensuring fairness, emphasizing the importance of independent investigative bodies for incidents of research misconduct, and advocating for fairness as a priority in governance of research misconduct.

Factors influencing the promotion and implementation of research integrity in research performing and research funding organizations: A scoping review

Promoting and implementing research integrity is considered the joint responsibility and effort of multiple stakeholders in the research community. We conducted a scoping review and analyzed 236 research articles and gray literature publications from biomedical sciences, social sciences, natural sciences (including engineering), and humanities that dealt with the factors that may positively or negatively impact the promotion and implementation of research integrity. Critical appraisal of evidence was performed for studies describing interventions aimed at research integrity promotion in order to provide insight into the effectiveness of these interventions. The results of this scoping review provide a comprehensive taxonomy of factors with positive or negative impact and their relatedness to individual researchers, research performing and funding organizations, and the system of science. Moreover, the results show that efforts for fostering and promoting research integrity should be implemented at all three levels (researcher, institution, system) simultaneously to deliver greater adherence and implementation of research integrity practices. Although various educational interventions aiming at research integrity promotion exist, we were not able to conclude on the effectiveness of explored interventions due to the methodological quality issues in the studies.

Superb supervision: A pilot study on training supervisors to convey responsible research practices onto their PhD candidates

One way to strengthen research integrity, is through supervision. According to previous research, a supervisor should be well-versed in responsible research practices (RRPs) and possess the necessary interpersonal skills to convey RRPs. We developed a 3-day pilot training for PhD supervisors that combined RRPs and interpersonal skills. Our aim was to assess: perceptions regarding supervision skills (before and after the pilot) and participants' views on combining RRPs and interpersonal skills. Before and after the pilot, we sent the Research Supervision Quality Evaluation survey to the participating PhD supervisors and their PhD candidates. The pilot was concluded with a focus group where participants deliberated over the combination of training in interpersonal skills and RRPs and whether such training should become compulsory. Both supervisors and PhD candidates were more positive about the supervisor's interpersonal skills and the ability to foster RRPs after the training. Participants were enthusiastic about the training's dual focus but believed that making the training compulsory would be undesirable. The results highlight the potential of RRPs training for supervisors. However, caution is warranted, as the results regard a small sample of volunteering supervisors, underscoring the need for larger programs to foster responsible supervision that are rigorously evaluated.

The Mentor's Role in Fostering Research Integrity Standards Among New Generations of Researchers: A Review of Empirical Studies

Promoting research integrity practices among doctoral candidates and early career researchers is important for creating a stable and healthy research environment. In addition to teaching specific technical skills and knowledge, research supervisors and mentors inevitably convey research practices, both directly and indirectly. We conducted a scoping review to summarise the role of mentors in fostering research integrity practices, mentors' responsibilities and the role that institutions have in supporting good mentorship. We searched five different databases and included studies that used an empirical methodology. After searching, a total of 1199 articles were retrieved, of which 24 were eligible for analysis. After snowballing, a total of 35 empirical articles were selected. The review discusses various themes such as the importance of good mentorship, poor mentorship practices, virtues and qualities of mentors, responsibilities and activities of mentors, group mentoring and responsibilities of the institution in supporting good mentorship. This review demonstrates the importance of mentors instilling responsible research practices and attitudes, and promoting research integrity among their mentees. Mentors are responsible for providing explicit guidance and for acting as good role models. The review highlights how poor mentorship can have a bad impact on the research climate. In addition, the review highlights the important influence that institutions can have in supporting mentorship.

A study on the content of integrity policies and research integrity management in Chinese universities

Background

This study outlines a comprehensive analysis of the primary characteristics of managing research integrity (RI) in domestic colleges and universities in China. RI education in China consists primarily of soft advocacy, with no hard requirements or continuous and systematic support. Together with other stakeholders, such as funders and publishers, higher education institutions (e.g., colleges and universities) are one of the vital actors that have a lot of influence on RI promotion and implementation among researchers. However, the literature on the regulation of RI policies in China's universities is limited.

Methods

We investigate the top 50 colleges and universities in the 2021 Best Chinese Universities Ranking. Their guidance and policy documents on RI were collected via their official websites. By integrating the use of scientometrics analysis, including descriptive statistical analysis, inductive content analysis, and quantitative analysis, we examine whether and how these higher education institutions respond to national policies in a timely manner, especially in terms of their frequency of updates, topic clustering analysis, terms clustering analysis, content aggregation. To further understand the composition mechanism and the main working systems of university RI management organizations, we conducted in-depth research on the organizational functions, meeting system, staff composition mechanism, and scientific research misconduct acceptance and investigation mechanisms.

Results

The regulations on the treatment of RI in China's universities have, in response to the government's call to establish their own management policies and working mechanisms, maintained a zero-tolerance stance on research misconduct. The sampled universities listed the definition and principles of misconduct practices, investigation procedures, and sanctions of research misconduct in their own policy documents. Some of them listed inappropriate research practices All 50 sampled universities have formed relevant organizations responsible for RI management, they all provide the detailed regulations of the committees. Yet, there is still a need to further define Questionable Research Practice, foster higher standards for integrity in research and, establish and improve an efficient, authoritative, well-restrained and supervision working mechanism for organizations responsible for RI treatment.

“It is Very Difficult for us to Separate Ourselves from this System”: Views of European Researchers, Research Managers, Administrators and Governance Advisors on Structural and Institutional Influences on Research Integrity

Research integrity is fundamental to the validity and reliability of scientific findings, and for ethical conduct of research. As part of PRINTEGER (Promoting Integrity as an Integral Dimension of Excellence in Research), this study explores the views of researchers, research managers, administrators, and governance advisors in Estonia, Italy, Norway and UK, focusing specifically on their understanding of institutional and organisational influences on research integrity.

A total of 16 focus groups were conducted. Thematic analysis of the data revealed that competition is pervasive and appeared in most themes relating to integrity. The structural frameworks for research such as funding, evaluation and publication were thought to both protect and, more commonly, undermine integrity. In addition, institutional systems, including workload and research governance, shaped participants’ day-to-day work environment, also affecting research integrity. Participants also provided ideas for promoting research integrity, including training, and creating conditions that would be supportive of research integrity.

These findings support a shift away from individual blame and towards the need for structural and institutional changes, including organisations in the wider research environment, for example funding bodies and publishing companies.

Leaving academia: PhD attrition and unhealthy research environments

This study investigates PhD candidates’ (N = 391) perceptions about their research environment at a Dutch university in terms of the research climate, (un)ethical supervisory practices, and questionable research practices. We assessed whether their perceptions are related to career considerations. We gathered quantitative self-report estimations of the perceptions of PhD candidates using an online survey tool and then conducted descriptive and within-subject correlation analysis of the results. While most PhD candidates experience fair evaluation processes, openness, integrity, trust, and freedom in their research climate, many report lack of time and support, insufficient supervision, and witness questionable research practices. Results based on Spearman correlations indicate that those who experience a less healthy research environment (including experiences with unethical supervision, questionable practices, and barriers to responsible research), more often consider leaving academia and their current PhD position.

Perception and reaction of Nanyang Technological University (NTU) researchers to different forms of research integrity education modality

BACKGROUND

Research and academic institutions use various delivery channels to deliver Research Integrity (RI) education in their communities. Yet there is no consensus on the best delivery method and the effectiveness of these channels in inculcating a positive RI culture varies across institutions. Hence, this study aimed to understand the preferences of the research community in Nanyang Technological University (NTU), Singapore.

METHODS

An online survey was conducted on NTU research community to understand their experience with, and preference for each RI education mode offered in NTU. The RI education modes surveyed in the general ranking question are Data Management Plan (DMP) workshops, Epigeum e-Learning, Compass e-newsletter (email), and NTU policy on Research Integrity and Responsible Conduct of Research. There were 242 responses, comprising 50% research students, 32.2% research staff and 17.8% faculty members. Non-parametric statistical techniques were used to analyse preferences across different RI education modes and within sub-groups (i.e., fields, age, native language, roles in research community).

RESULTS

More than 92% of respondents subscribed to the importance of RI education, but with different preferences for education modes. With respect to RI education in NTU, Compass e-newsletters were ranked the lowest (p < 0.05). Most felt that they were too wordy and unengaging, making it difficult to absorb information. Similarly, Epigeum e-Learning (p < 0.05) and 'policy' (p < 0.05) were found to be too lengthy in presentation. The compulsory NTU RI education modes (Epigeum e-learning and 'policy') enjoyed higher participation rates of 70-80% compared with 32-37% for the self-regulated modes (DMP workshop and e-newsletter). This suggests that regulatory mechanisms are still necessary to promote participation in RI education, and thus, core RI education content should be made compulsory in research/academic institutions. Although Epigeum is a compulsory course, some may not have participated in the programme due to technical issues or they might have forgotten to participate in the programme within the permissible timeframe. For all four RI education modes in NTU, the lack of awareness was among the top cited reasons for not participating.

CONCLUSIONS

Most NTU researchers perceived RI education positively although they may have reservations for some approaches. Conversely, e-Learning is favored over all the other modes except for the mode of Policy. Findings from this study are useful for improving the design of RI education strategies to be more appealing to the research community by enhancing user experience in terms of user-friendliness, relevance to specialisation, providing concise information and better presentation of materials For institutions with similar modes of RI education as NTU, these results may be relevant in improving participation rates and presentation of RI education modes, such as the use of infographics and more concise information.

Focused Research on the Challenges and Productivity of Researchers in Nigerian Academic Institutions Without Funding

Background: The challenge of research funding constraints has brought to bear enormous pressure on researchers. Research productivity is relevant to prestige and career progression of academic staff. However, this study aimed to explore significant challenges associated with researchers' productivity and the impact of non-funding of research in Nigerian research and tertiary institutions. Methods: This study adopted a qualitative exploratory design involving academics at various research and tertiary institutions across the six geographical regions in Nigeria. A semi-structured questionnaire was distributed electronically to all participants who consented to take part in this study. Exactly 4,159 questionnaires were administered and 2,350 were completely filled and returned. Pearson correlation matrices with logistic regression were used for data analysis and are presented in frequencies and percentages. Results: On challenges faced by respondents, 42.98% reported a lack of research funding, 17.11% mentioned brain drain challenge while 8.85% indicated a lack of motivation. Of the 23,927 publications reported, the number of those in sciences, engineering, and medical sciences averaged 9.6, 11.5, and 9.5 respectively. The average number of publications by women (10.8) was more than by men (9.7). Lecturers had the highest average research publication number (11.8) followed by researchers (10.2) and others (3.9). Men had the highest (11.9) average number of conferences compared to women (9.2). Participants in engineering had an average number of 13.8 conferences per respondents followed by those in education (11.2), sciences (11.1), and 10.9 for those in agricultural sciences. The result revealed a negative significant correlation between research publication and academic qualification at p < 0.01. Positive significant correlation was observed between research productivity and discipline at p < 0.05. Findings show that the combined influence of the independent variables on research productivity was significant using linear regression analysis. Conclusions: The failure to prioritize research has resulted in underdevelopment in Nigeria. It is therefore imperative that the federal government prioritize research and establish a functional Special Research Trust Fund to oversee research funding in Nigeria.

Reflexive Behaviour: How Publication Pressure Affects Research Quality in Astronomy

Reflexive metrics is a branch of science studies that explores how the demand for accountability and performance measurement in science has shaped the research culture in recent decades. Hypercompetition and publication pressure are part of this neoliberal culture. How do scientists respond to these pressures? Studies on research integrity and organisational culture suggest that people who feel treated unfairly by their institution are more likely to engage in deviant behaviour, such as scientific misconduct. By building up on reflexive metrics, combined with studies on the influence of organisational culture on research integrity, this study reflects on the research behaviour of astronomers with the following questions: (1) To what extent is research (mis-)behaviour reflexive, i.e., dependent on perceptions of publication pressure and distributive and organisational justice? (2) What impact does scientific misconduct have on research quality? In order to perform this reflection, we conducted a comprehensive survey of academic and non-academic astronomers worldwide and received 3509 responses. We found that publication pressure explains 10% of the variance in occurrence of misconduct and between 7% and 13% of the variance of the perception of distributive and organisational justice as well as overcommitment to work. Our results on the perceived impact of scientific misconduct on research quality show that the epistemic harm of questionable research practices should not be underestimated. This suggests there is a need for a policy change. In particular, lesser attention to metrics (such as publication rate) in the allocation of grants, telescope time and institutional rewards would foster better scientific conduct and, hence, research quality.

Grant writing and grant peer review as questionable research practices

A large part of governmental research funding is currently distributed through the peer review of project proposals. In this paper, we argue that such funding systems incentivize and even force researchers to violate five moral values, each of which is central to commonly used scientific codes of conduct. Our argument complements existing epistemic arguments against peer-review project funding systems and, accordingly, strengthens the mounting calls for reform of these systems.

Grant writing and grant peer review as questionable research practices

A large part of governmental research funding is currently distributed through the peer review of project proposals. In this paper, we argue that such funding systems incentivize and even force researchers to violate five moral values, each of which is central to commonly used scientific codes of conduct. Our argument complements existing epistemic arguments against peer-review project funding systems and, accordingly, strengthens the mounting calls for reform of these systems.

Development and implementation of research integrity guidance documents: Explorative interviews with research integrity experts

Research integrity (RI) guidance documents often lack sufficient details on handling specific RI issues causing the lack of harmonized approaches to RI and opening the way to research misconduct and other detrimental research practices. Standard operating procedures (SOPs) are developed and implemented by organizations for ensuring the uniformity and quality of performed actions. This study aimed to explore stakeholders' opinions on SOPs for RI, factors influencing the implementation of RI guidance documents and practices, and ideas for improvements in the RI field. We conducted semi-structured interviews with stakeholders from different groups. Data were analyzed using the reflexive thematic analysis approach, and three themes were developed. The first theme addressed participants' knowledge and perceptions on SOPs for RI and their impact on RI promotion and implementation. The second theme described different factors that have a positive or negative impact on the implementation of RI and RI guidance documents and practices, while the third theme addressed needed changes and ideas for improvements in the RI field. Participants considered SOPs valuable for RI promotion. SOPs should be developed based on and consistent with more general and aspirational guidance and through the dialogue with researchers and other stakeholders, to ensure their relevancy.

Awareness Mentality and Strategic Behavior in Science

Acknowledgement of scientific achievements was and is essentially achieved through the citation of a publication. Increasingly, however, it is no longer just the publication itself that plays an important role, but also the degree of attention that a scientist achieves with this very publication. Thus, the importance of strategic behavior in science is progressing and an awareness mentality is spreading. In this paper, the causes and backgrounds of this development are discussed, identifying the use of reductionist, quantitative systems in science management and research funding, the loss of critical judgment and technocratic dominance, quantitative assessments used for decision making, altmetrics and the like as alternative views, the use of perception scores in reference databases and universities as well as ambitions of journals as main drivers. Besides, different forms of strategic behavior in science and the resulting consequences and impacts are being highlighted.

Survey on the Research Misconduct and Questionable Research Practices of Medical Students, PhD Students, and Supervisors at the Zagreb School of Medicine in Croatia

This cross-sectional study evaluates the knowledge, attitudes, experiences, and behavior of final year medical students, PhD students, and supervisors at the School of Medicine of the University of Zagreb in relation to research misconduct, questionable research practices, and the research environment. The overall response rate was 36.4% (68%-100% for the paper survey and 8%-15% for the online surveys). The analysis reveals statistically significant differences in attitude scores between PhD students and supervisors, the latter having attitudes more in concordance with accepted norms. The results overall show a nonnegligible incidence of self-reported misconduct and questionable research practices, as well as some problematic attitudes towards misconduct and questionable research practices. The incidence of problematic authorship practices was particularly high. The research environment was evaluated as being mostly supportive of research integrity.

Towards a Research Agenda for Promoting Responsible Research Practices

This opinion piece aims to inform future research funding programs on responsible research practices (RRP) based on three specific objectives: (1) to give a sketch of the current international discussion on responsible research practices (RRPs); (2) to give an overview of current initiatives and already obtained results regarding RRP; and (3) to give an overview of potential future needs for research on RRP. In this opinion piece, we have used seven iterative methodological steps (including literature review, ranking, and sorting exercises) to create the proposed research agenda. We identified six main themes that we believe need attention in future research: (1) responsible evaluation of research and researchers, (2) the influence of open science and transparency on RRP, (3) research on responsible mentoring, supervision, and role modeling, (4) the effect of education and training on RRP, (5) checking for reproducibility, and (6) responsible and fair peer review. These themes have in common that they address aspects of research that are mostly on the level of the scientific system, more than on the level of the individual researcher. Some current initiatives are already gathering substantial empirical evidence to start filling these gaps. We believe that with sufficient support from all relevant stakeholders, more progress can be made.

Reporting health services research to a broader public: An exploration of inconsistencies and reporting inadequacies in societal publications

Introduction

Little is known about the accuracy of societal publications (e.g. press releases, internet postings or professional journals) that are based on scientific work. This study investigates a) inconsistencies between scientific peer-reviewed health services research (HSR) publications and non-scientific societal publications and b) replication of reporting inadequacies from these scientific publications to corresponding societal publications.

Methods

A sample of HSR publications was drawn from 116 publications authored in 2016 by thirteen Dutch HSR institutions. Societal publications corresponding to scientific publications were identified through a systematic internet search. We conducted a qualitative, directed content analysis on societal publications derived from the scientific publications to assess both reporting inadequacies and determine inconsistencies. Descriptive frequencies were calculated for all variables. Odds ratios were used to investigate whether inconsistencies in societal publications were less likely when the first scientific author was involved.

Results

We identified 43 scientific and 156 societal publications. 94 societal publications (60.3%), (associated with 32 scientific publications (74.4%)) contained messages that were inconsistent with the scientific work. We found reporting inadequacies in 22 scientific publications (51.2%). In 45 societal publications (28.9%), we found replications of these reporting inadequacies. The likelihood of inconsistencies between scientific and societal publications did not differ when the latter explicitly involved the first scientific author, (OR = 1.44, CI: 0.76–2.74); were published on the institute’s or funder’s website, (OR = 1.32, CI: 0.57–3.06); published with no involvement of a scientific author, (OR = 0.52, CI: 0.25–1.07).

Conclusion

To improve societal publications, one should examine both the consistency with scientific research publications and ways to prevent replication of scientific reporting inadequacies. HSR institutions, funders, and scientific and societal publication platforms should invest in a supportive publication culture to further incentivise the responsible and skilled involvement of researchers in writing both scientific and societal publications.

Interventions for Organizational Climate and Culture in Academia: A Scoping Review

Organizational climate and culture may influence different work-related outcomes, including responsible conduct of research and research misconduct in academic or research organizations. In this scoping review we collected evidence on outcomes of interventions to change organizational climate or culture in academic or research settings. Out of 32,093 documents retrieved by the search, we analysed 207 documents in full text, out of which 7 met the eligibility criteria and were included in the final analysis. The included studies measured organizational climate (2 studies), organizational culture (4 studies), or both (1 study) at biomedical faculties (4 studies) or non-academic university departments (3 studies). Four studies had post-test, and three before-and-after study designs. The majority of interventions were face-to-face activities (meetings, different teambuilding activities), and two were based on organizational change. Six studies reported positive changes in organizational climate/culture after the intervention. These positive changes were measured as improvements in score on different questionnaire survey or were described through authors' or external evaluator's narrative reports. However, the methodological quality of the studies was low, both for qualitative and quantitative study designs. Replicable studies, using rigorous methods and clearly defined outcomes are urgently needed if organizations want to achieve a real change in organizational climate or culture for responsible research. The protocol for this scoping review was registered at https://osf.io/7zjqb .

Expanding Research Integrity: A Cultural-Practice Perspective

Research integrity (RI) is usually discussed in terms of responsibilities that individual researchers bear towards the scientific work they conduct, as well as responsibilities that institutions have to enable those individual researchers to do so. In addition to these two bearers of responsibility, a third category often surfaces, which is variably referred to as culture and practice. These notions merit further development beyond a residual category that is to contain everything that is not covered by attributions to individuals and institutions. This paper discusses how thinking in RI can take benefit from more specific ideas on practice and culture. We start by articulating elements of practice and culture, and explore how values central to RI are related to these elements. These insights help identify additional points of intervention for fostering responsible conduct. This helps to build "cultures and practices of research integrity", as it makes clear that specific times and places are connected to specific practices and cultures and should have a place in the debate on Research Integrity. With this conceptual framework, practitioners as well as theorists can avoid using the notions as residual categories that de facto amount to vague, additional burdens of responsibility for the individual.

Spanish Scientists' Opinion about Science and Researcher Behavior

We surveyed 348 Psychology and Education researchers within Spain, on issues such as their perception of a crisis in Science, their confidence in the quality of published results, and the use of questionable research practices (QRP). Their perceptions regarding pressure to publish and academic competition were also collected. The results indicate that a large proportion of the sample of Spanish academics think there is a crisis in Science, mainly due to a lack of economic investment, and doubts the quality of published findings. They also feel strong pressure to publish in high impact factor journals and a highly competitive work climate.

Citation bias and other determinants of citation in biomedical research: findings from six citation networks

OBJECTIVES

When the probability of being cited depends on the outcome of that study, this is called citation bias. The aim of this study is to assess the determinants of citation and how these compare across six different biomedical research fields.

STUDY DESIGN AND SETTING

Citation network analyses were performed for six biomedical research questions. After identifying all relevant publications, all potential citations were mapped together with the actually performed citations in each network. As determinants of citation we assessed the following: study outcome, study design, sample size, journal impact factor, gender, affiliation, authority and continent of the corresponding author, funding source, title of the publication, number of references, and self-citation. Random effect logistic regression analysis was used to assess these factors.

RESULTS

Four out of six networks showed evidence for citation bias. Self-citation, authority of the author, and journal impact factor were also positively associated with the probability of citation in all networks.

CONCLUSION

The probability of being cited seems associated with positive study outcomes, the authority of its authors, and the journal in which that article is published. In addition, each network showed specific characteristics that impact the citation dynamics and that need to be considered when performing and interpreting citation analyses.

Researchers' Perceptions of a Responsible Research Climate: A Multi Focus Group Study

The research climate plays a key role in fostering integrity in research. However, little is known about what constitutes a responsible research climate. We investigated academic researchers' perceptions on this through focus group interviews. We recruited researchers from the Vrije Universiteit Amsterdam and the Amsterdam University Medical Center to participate in focus group discussions that consisted of researchers from similar academic ranks and disciplinary fields. We asked participants to reflect on the characteristics of a responsible research climate, the barriers they perceived and which interventions they thought fruitful to improve the research climate. Discussions were recorded and transcribed at verbatim. We used inductive content analysis to analyse the focus group transcripts. We conducted 12 focus groups with 61 researchers in total. We identified fair evaluation, openness, sufficient time, integrity, trust and freedom to be mentioned as important characteristics of a responsible research climate. Main perceived barriers were lack of support, unfair evaluation policies, normalization of overwork and insufficient supervision of early career researchers. Possible interventions suggested by the participants centered around improving support, discussing expectations and improving the quality of supervision. Some of the elements of a responsible research climate identified by participants are reflected in national and international codes of conduct, such as trust and openness. Although it may seem hard to change the research climate, we believe that the realisation that the research climate is suboptimal should provide the impetus for change informed by researchers' experiences and opinions.

Individual, institutional, and scientific environment factors associated with questionable research practices in the reporting of messages and conclusions in scientific health services research publications

Background

Health Services Research findings (HSR) reported in scientific publications may become part of the decision-making process on healthcare. This study aimed to explore associations between researcher’s individual, institutional, and scientific environment factors and the occurrence of questionable research practices (QRPs) in the reporting of messages and conclusions in scientific HSR publications.

Methods

We employed a mixed-methods study design. We identified factors possibly contributing to QRPs in the reporting of messages and conclusions through a literature review, 14 semi-structured interviews with HSR institutional leaders, and 13 focus-groups amongst researchers. A survey corresponding with these factors was developed and shared with 172 authors of 116 scientific HSR publications produced by Dutch research institutes in 2016. We assessed the included publications for the occurrence of QRPs. An exploratory factor analysis was conducted to identify factors within individual, institutional, and environmental domains. Next, we conducted bivariate analyses using simple Poisson regression to explore factors’ association with the number of QRPs in the assessed HSR publications. Factors related to QRPs with a p-value < .30 were included in four multivariate models tested through a multiple Poisson regression.

Results

In total, 78 (45%) participants completed the survey (51.3% first authors and 48.7% last authors). Twelve factors were included in the multivariate analyses. In all four multivariate models, a higher score of “pressure to create societal impact” (Exp B = 1.28, 95% CI [1.11, 1.47]), was associated with higher number of QRPs. Higher scores on “specific training” (Exp B = 0.85, 95% CI [0.77–0.94]) and “co-author conflict of interest” (Exp B = 0.85, 95% CI [0.75–0.97]) factors were associated with a lower number of QRPs. Stratification between first and last authors indicated different factors were related to the occurrence of QRPs for these groups.

Conclusion

Experienced pressure to create societal impact is associated with more QRPs in the reporting of messages and conclusions in HSR publications. Specific training in reporting messages and conclusions and awareness of co-author conflict of interests are related to fewer QRPs. Our results should stimulate awareness within the field of HSR internationally on opportunities to better support reporting in scientific HSR publications.

Professionalism in Science: Competence, Autonomy, and Service

Some of the most significant policy responses to cases of fraudulent and questionable conduct by scientists have been to strengthen professionalism among scientists, whether by codes of conduct, integrity boards, or mandatory research integrity training programs. Yet there has been little systematic discussion about what professionalism in scientific research should mean. In this paper I draw on the sociology of the professions and on data comparing codes of conduct in science to those in the professions, in order to examine what precisely the model of professionalism implies for scientific research. I argue that professionalism, more than any other single organizational logic, is appropriate for scientific research, and that codes of conduct for scientists should strengthen statements concerning scientific autonomy and competence, as well as the scientific service ideal.

Collecting and characterizing existing and freely accessible research integrity educational resources

In addition to effective training practices, well-structured educational resources are important for developing successful research integrity training programs. A considerable amount of educational material has been developed in the last years, but there is a necessity to find better ways to assess and categorize the already existing resources. We collected 237 freely available online RI educational resources with the aim to describe them in as much detail as possible using a set of well-defined criteria. We developed a grid that gives a full description, based on our 21 criteria, for each collected resource. Mainly videos and online RI training are present in our collection. Worldwide, resources are mainly from the US, whereas in Europe mainly from the UK. In the majority of the cases, the educational resources are not customized, presenting the big three (falsification, fabrication, and plagiarism) as the most addressed topics. Making RI educational resources easily accessible might help to increase awareness about the topic. Moreover, the characterization we provide might help researchers and students to deal with daily RI-related issues, to look for the right tool at the right time, and might help institutions and trainers to develop new trainings without the need to develop new tools.Abbreviations: CITI: Collaborative Institutional Training Initiative; COPE: Committee on Publication Ethics; ENERI: European Network of Research Ethics and Research Integrity; ENRIO: the European Network of Research Integrity Offices; EU: European Union; NIH: National Institutes of Health; NSF: National Science Foundation; NRIN: the Netherlands Research Integrity Network; ORI: the Office of Research Integrity; PPT: powerpoint; QRP: questionable research practice; RI: research integrity; RCR: responsible conduct of research.

Responsible scientific research at Inserm: a field study

In France, Inserm (National Institute of Health And Medical Research) puts considerable effort into scientific integrity and quality management since more than 10 yr. Quality managers are present in about one hundred of the three hundred Inserm Units (the “Unit” at Inserm is an individualized research laboratory, with a head which is nominated by the President of Inserm and a defined life span. Usually, Units are the result of partnerships of Inserm with universities, university hospitals and other research organizations). When Quality management audits are performed, various dysfunctions are noted. Some of these are related to scientific integrity and Questionable Research Practice. Since an increasing number of Inserm Units disposes of Quality management systems according to the ISO 9001 standard, Inserm promotes since several years ISO 9001 management concepts and tools to improve scientific integrity. “Scientific integrity” as used in the missions of the (American) Office of Research Integrity (ORI, https://ori.hhs.gov/policies-ori-mission) and materialized in the Singapore statement on Research integrity (https://wcrif.org/statement). The Singapore Statement is not a regulatory document and does not represent the official policies of the countries represented at the conference. Rather, the intent of the Singapore Statement is to provide ethical guidance which research organizations, governments, and scientists can use to develop policies, regulations, and codes of conduct (World Conference on Research Integrity, https://wcrif.org/statement). Inserm, like others (Bouter, Account. Res. 22, 148–161 (2015)), believes that ISO 9001 research management favors good quality and integrity of research. In this field study the degree of awareness and whether scientists in Inserm Units are ready to make specific efforts to foster integrity and quality of their research was investigated. This study shows that a majority of scientists considers integrity and quality as being necessary. However, it appears that knowledge and understanding of integrity and quality in research should be improved. An effort has to be made to increase this level, specific actions have to be taken but scientists at Inserm are willing to make this effort.

Ethical Consistency and Experience: An Attempt to Influence Researcher Attitudes Toward Questionable Research Practices Through Reading Prompts

Over the past couple of decades, the apparent widespread occurrence of Questionable Research Practices (QRPs) in scientific research has been widely discussed in the research ethics literature as a source of concern. Various ways of reducing their use have been proposed and implemented, ranging from improved training and incentives for adopting best practices to systematic reforms. This article reports on the results of two studies that investigated the efficacy of simple, psychological interventions aimed at changing researcher attitudes toward QRPs. While the interventions did not significantly modify researchers’ reactions to QRPs, they showed differential efficacy depending on scientists’ experience, suggesting complexities in researcher psychology and the ethics of QRPs that merit further study.

Researchers' perceptions of research misbehaviours: a mixed methods study among academic researchers in Amsterdam

Background

There is increasing evidence that research misbehaviour is common, especially the minor forms. Previous studies on research misbehaviour primarily focused on biomedical and social sciences, and evidence from natural sciences and humanities is scarce. We investigated what academic researchers in Amsterdam perceived to be detrimental research misbehaviours in their respective disciplinary fields.

Methods

We used an explanatory sequential mixed methods design. First, survey participants from four disciplinary fields rated perceived frequency and impact of research misbehaviours from a list of 60. We then combined these into a top five ranking of most detrimental research misbehaviours at the aggregate level, stratified by disciplinary field. Second, in focus group interviews, participants from each academic rank and disciplinary field were asked to reflect on the most relevant research misbehaviours for their disciplinary field. We used participative ranking methodology inducing participants to obtain consensus on which research misbehaviours are most detrimental.

Results

In total, 1080 researchers completed the survey (response rate: 15%) and 61 participated in the focus groups (3 three to 8 eight researchers per group). Insufficient supervision consistently ranked highest in the survey regardless of disciplinary field and the focus groups confirmed this. Important themes in the focus groups were insufficient supervision, sloppy science, and sloppy peer review. Biomedical researchers and social science researchers were primarily concerned with sloppy science and insufficient supervision. Natural sciences and humanities researchers discussed sloppy reviewing and theft of ideas by reviewers, a form of plagiarism. Focus group participants further provided examples of particular research misbehaviours they were confronted with and how these impacted their work as a researcher.

Conclusion

We found insufficient supervision and various forms of sloppy science to score highly on aggregate detrimental impact throughout all disciplinary fields. Researchers from the natural sciences and humanities also perceived nepotism to be of major impact on the aggregate level. The natural sciences regarded fabrication of data of major impact as well. The focus group interviews helped to understand how researchers interpreted ‘insufficient supervision’. Besides, the focus group participants added insight into sloppy science in practice. Researchers from the natural sciences and humanities added new research misbehaviours concerning their disciplinary fields to the list, such as the stealing of ideas before publication. This improves our understanding of research misbehaviour beyond the social and biomedical fields.

Perceived publication pressure in Amsterdam: Survey of all disciplinary fields and academic ranks

Publications determine to a large extent the possibility to stay in academia ("publish or perish"). While some pressure to publish may incentivise high quality research, too much publication pressure is likely to have detrimental effects on both the scientific enterprise and on individual researchers. Our research question was: What is the level of perceived publication pressure in the four academic institutions in Amsterdam and does the pressure to publish differ between academic ranks and disciplinary fields? Investigating researchers in Amsterdam with the revised Publication Pressure Questionnaire, we find that a negative attitude towards the current publication climate is present across academic ranks and disciplinary fields. Postdocs and assistant professors (M = 3.42) perceive the greatest publication stress and PhD-students (M = 2.44) perceive a significant lack of resources to relieve publication stress. Results indicate the need for a healthier publication climate where the quality and integrity of research is rewarded.

Occurrence and nature of questionable research practices in the reporting of messages and conclusions in international scientific Health Services Research publications: a structured assessment of publications authored by researchers in the Netherlands

OBJECTIVES

Explore the occurrence and nature of questionable research practices (QRPs) in the reporting of messages and conclusions in international scientific Health Services Research (HSR) publications authored by researchers from HSR institutions in the Netherlands.

DESIGN

In a joint effort to assure the overall quality of HSR publications in the Netherlands, 13 HSR institutions in the Netherlands participated in this study. Together with these institutions, we constructed and validated an assessment instrument covering 35 possible QRPs in the reporting of messages and conclusions. Two reviewers independently assessed a random sample of 116 HSR articles authored by researchers from these institutions published in international peer-reviewed scientific journals in 2016.

SETTING

Netherlands, 2016.

SAMPLE

116 international peer-reviewed HSR publications.

MAIN OUTCOME MEASURES

Median number of QRPs per publication, the percentage of publications with observed QRP frequencies, occurrence of specific QRPs and difference in total number of QRPs by methodological approach, type of research and study design.

RESULTS

We identified a median of six QRPs per publication out of 35 possible QRPs. QRPs occurred most frequently in the reporting of implications for practice, recommendations for practice, contradictory evidence, study limitations and conclusions based on the results and in the context of the literature. We identified no differences in total number of QRPs in papers based on different methodological approach, type of research or study design.

CONCLUSIONS

Given the applied nature of HSR, both the severity of the identified QRPs, and the recommendations for policy and practice in HSR publications warrant discussion. We recommend that the HSR field further define and establish its own scientific norms in publication practices to improve scientific reporting and strengthen the impact of HSR. The results of our study can serve as an empirical basis for continuous critical reflection on the reporting of messages and conclusions.

Perceptions of research integrity climate differ between academic ranks and disciplinary fields: Results from a survey among academic researchers in Amsterdam

Breaches of research integrity have shocked the academic community. Initially explanations were sought at the level of individual researchers but over time increased recognition emerged of the important role that the research integrity climate may play in influencing researchers' (mis)behavior. In this study we aim to assess whether researchers from different academic ranks and disciplinary fields experience the research integrity climate differently. We sent an online questionnaire to academic researchers in Amsterdam using the Survey of Organizational Research Climate. Bonferroni corrected mean differences showed that junior researchers (PhD students, postdocs and assistant professors) perceive the research integrity climate more negatively than senior researchers (associate and full professors). Junior researchers note that their supervisors are less committed to talk about key research integrity principles compared to senior researchers (MD = -.39, CI = -.55, -.24). PhD students perceive more competition and suspicion among colleagues (MD = -.19, CI = -.35, -.05) than associate and full professors. We found that researchers from the natural sciences overall express a more positive perception of the research integrity climate. Researchers from social sciences as well as from the humanities perceive less fairness of their departments' expectations in terms of publishing and acquiring funding compared to natural sciences and biomedical sciences (MD = -.44, CI = -.74, -.15; MD = -.36, CI = -.61, -.11). Results suggest that department leaders in the humanities and social sciences should do more to set fairer expectations for their researchers and that senior scientists should ensure junior researchers are socialized into research integrity practices and foster a climate in their group where suspicion among colleagues has no place.

Researcher Requests for Inappropriate Analysis and Reporting: A U.S. Survey of Consulting Biostatisticians

BACKGROUND

Inappropriate analysis and reporting of biomedical research remain a problem despite advances in statistical methods and efforts to educate researchers.

OBJECTIVE

To determine the frequency and severity of requests biostatisticians receive from researchers for inappropriate analysis and reporting of data during statistical consultations.

DESIGN

Online survey.

SETTING

United States.

PARTICIPANTS

A randomly drawn sample of 522 American Statistical Association members self-identifying as consulting biostatisticians.

MEASUREMENTS

The Bioethical Issues in Biostatistical Consulting Questionnaire soliciting reports about the frequency and perceived severity of specific requests for inappropriate analysis and reporting.

RESULTS

Of 522 consulting biostatisticians contacted, 390 provided sufficient responses: a completion rate of 74.7%. The 4 most frequently reported inappropriate requests rated as "most severe" by at least 20% of the respondents were, in order of frequency, removing or altering some data records to better support the research hypothesis; interpreting the statistical findings on the basis of expectation, not actual results; not reporting the presence of key missing data that might bias the results; and ignoring violations of assumptions that would change results from positive to negative. These requests were reported most often by younger biostatisticians.

LIMITATIONS

The survey provides information on the reported frequency of inappropriate requests but not on how such requests were handled or whether the requests reflected researchers' maleficence or inadequate knowledge about statistical and research methods. In addition, other inappropriate requests may have been made that were not prespecified in the survey.

CONCLUSION

This survey suggests that researchers frequently make inappropriate requests of their biostatistical consultants regarding the analysis and reporting of their data. Understanding the reasons for these requests and how they are handled requires further study.

PRIMARY FUNDING SOURCE

U.S. Department of Health and Human Services.

Is failure to raise concerns about misconduct a breach of integrity? Researchers' reflections on reporting misconduct

This article describes and discusses the views of researchers on the significance of raising concerns about scientific misconduct in their work environment and the reasons or circumstances that might deter them from doing so. In this exploratory qualitative research study, we conducted in-depth interviews with 33 researchers working in life sciences and medicine. They represent three seniority levels and five universities across Switzerland. A large majority of respondents in this research study argued that failure to raise concerns about scientific misconduct compromises research integrity. This is an encouraging result demonstrating that researchers try to adhere to high ethical standards. However, further interaction with respondents highlighted that this correct ethical assessment does not lead researchers to take the consequent action of raising concerns. The factors that discourage researchers from raising concerns need to be addressed at the level of research groups, institutions, and by setting a positive precedent which helps them to believe in the system's ability to investigate concerns raised in a timely and professional manner. Training of researchers in research integrity related issues will have limited utility unless it is coupled with the creation of research culture where raising concerns is a standard practice of scientific and research activities.

Both Whistleblowers and the Scientists They Accuse Are Vulnerable and Deserve Protection

Whistleblowers play an important role diagnosing research misconduct, but often experience severe negative consequences. That is also true for incorrectly accused scientists. Both categories are vulnerable and deserve protection. Whistleblowers must proceed carefully and cautiously. Anonymous whistleblowing should be discouraged but cannot be ignored when the allegations are specific, serious, and plausible. When accused of a breach of research integrity it is important to be as transparent as possible. Sometimes accusations are false in the sense that the accuser knows or should know that the allegations are untrue. A mala fide whistleblower typically does not act carefully and we postulate a typology that may help in detecting them. Striking the right balance between whistleblower protection and timely unmasking false and identifying incorrect accusations is a tough dilemma leaders of research institutions have to face.

Research Ethics 2.0: New Perspectives on Norms, Values, and Integrity in Genomic Research in Times of Even Scarcer Resources

Research ethics anew gained importance due to the changing scientific landscape and increasing demands and competition in the academic field. These changes are further exaggerated because of scarce(r) resources in some countries on the one hand and advances in genomics on the other. In this paper, we will highlight the current challenges thereof to scientific integrity. To mark key developments in research ethics, we will distinguish between what we call research ethics 1.0 and research ethics 2.0. Whereas research ethics 1.0 focuses on individual integrity and informed consent, research ethics 2.0 entails social scientific integrity within a broader perspective of a research network. This research network can be regarded as a network of responsibilities in which every stakeholder involved has to jointly meet the ethical challenges posed to research.

Ranking major and minor research misbehaviors: results from a survey among participants of four World Conferences on Research Integrity

BACKGROUND

Codes of conduct mainly focus on research misconduct that takes the form of fabrication, falsification, and plagiarism. However, at the aggregate level, lesser forms of research misbehavior may be more important due to their much higher prevalence. Little is known about what the most frequent research misbehaviors are and what their impact is if they occur.

METHODS

A survey was conducted among 1353 attendees of international research integrity conferences. They were asked to score 60 research misbehaviors according to their views on and perceptions of the frequency of occurrence, preventability, impact on truth (validity), and impact on trust between scientists on 5-point scales. We expressed the aggregate level impact as the product of frequency scores and truth, trust and preventability scores, respectively. We ranked misbehaviors based on mean scores. Additionally, relevant demographic and professional background information was collected from participants.

RESULTS

Response was 17% of those who were sent the invitational email and 33% of those who opened it. The rankings suggest that selective reporting, selective citing, and flaws in quality assurance and mentoring are viewed as the major problems of modern research. The "deadly sins" of fabrication and falsification ranked highest on the impact on truth but low to moderate on aggregate level impact on truth, due to their low estimated frequency. Plagiarism is thought to be common but to have little impact on truth although it ranked high on aggregate level impact on trust.

CONCLUSIONS

We designed a comprehensive list of 60 major and minor research misbehaviors. Our respondents were much more concerned over sloppy science than about scientific fraud (FFP). In the fostering of responsible conduct of research, we recommend to develop interventions that actively discourage the high ranking misbehaviors from our study.

How to prepare a systematic review of economic evaluations for informing evidence-based healthcare decisions: a five-step approach (part 1/3)

INTRODUCTION

Systematic reviews of economic evaluations are useful for synthesizing economic evidence about health interventions and for informing evidence-based decisions. Areas covered: As there is no detailed description of the methods for performing a systematic review of economic evidence, this paper aims to provide an overview of state-of-the-art methodology. This is laid out in a 5-step approach, as follows: step 1) initiating a systematic review; step 2) identifying (full) economic evaluations; step 3) data extraction, risk of bias and transferability assessment; step 4) reporting results; step 5) discussion and interpretation of findings. Expert commentary: The paper aims to help inexperienced reviewers and clinical practice guideline developers, but also to be a resource for experts in the field who want to check on current methodological developments.