An Ethics of the System: Talking to Scientists About Research Integrity

How Competition for Funding Impacts Scientific Practice: Building Pre-fab Houses but no Cathedrals

In the research integrity literature, funding plays two different roles: it is thought to elevate questionable research practices (QRPs) due to perverse incentives, and it is a potential actor to incentivize research integrity standards. Recent studies, asking funders, have emphasized the importance of the latter. However, the perspective of active researchers on the impact of competitive research funding on science has not been explored yet. Here, I address this issue by conducting a series of group sessions with researchers in two different countries with different degrees of competition for funding, from three scientific fields (medical sciences, natural sciences, humanities), and in two different career stages (permanent versus temporary employment). Researchers across all groups experienced that competition for funding shapes science, with many unintended negative consequences. Intriguingly, these consequences had little to do with the type of QRPs typically being presented in the research integrity literature. Instead, the researchers pointed out that funding could result in predictable, fashionable, short-sighted, and overpromising science. This was seen as highly problematic: scientists experienced that the 'projectification' of science makes it more and more difficult to do any science of real importance: plunging into the unknown or addressing big issues that need a long-term horizon to mature. They also problematized unintended negative effects from collaboration and strategizing. I suggest it may be time to move away from a focus on QRPs in connection with funding, and rather address the real problems. Such a shift may then call for entirely different types of policy actions.

More ethics in the laboratory, please! Scientists' perspectives on ethics in the preclinical phase

In recent years there have been calls to improve ethics in preclinical research. Promoting ethics in preclinical research should consider the perspectives of scientists. Our study aims to explore researchers' perspectives on ethics in the preclinical phase. Using interviews and focus groups, we collected views on ethical issues in preclinical research from experienced (n = 11) and early-stage researchers (ESRs) (n = 14) working in a gene therapy and regenerative medicine consortium. A recurring theme among ESRs was the impact of health-related preclinical research on climate change. They highlighted the importance of strengthening ethics in relations within the scientific community. Experienced researchers were focused on technicalities of methods used in preclinical research. They stressed the need for more safeguards to protect the sensitive personal data they work with. Both groups drew attention to the importance of the social context of research and its social impact. They agreed that it is important to be socially responsible - to be aware of and be sensitive to the needs and views of society. This study helps to identify key ethical challenges and, when combined with more data, can ultimately lead to informed and evidence-based improvements to existing regulations.

How to combine rules and commitment in fostering research integrity?

Research integrity (RI) is crucial for trustworthy research. Rules are important in setting RI standards and improving research practice, but they can lead to increased bureaucracy; without commensurate commitment amongst researchers toward RI, they are unlikely to improve research practices. In this paper, we explore how to combine rules and commitment in fostering RI. Research institutions can govern RI using markets (using incentives), bureaucracies (using rules), and network processes (through commitment and agreements). Based on Habermas' Theory of Communicative Action, we argue that network processes, as part of the lifeworld, can legitimize systems - that is, market or bureaucratic governance modes. This can regulate and support RI practices in an efficient way. Systems can also become dominant and repress consensus processes. Fostering RI requires a balance between network, market and bureaucratic governance modes. We analyze the institutional response to a serious RI case to illustrate how network processes can be combined with bureaucratic rules. Specifically, we analyze how the Science Committee established at Tilburg University in 2012 has navigated different governance modes, resulting in a normatively grounded and efficient approach to fostering RI. Based on this case, we formulate recommendations to research institutions on how to combine rules and commitment.

Empowering the Research Community to Investigate Misconduct and Promote Research Integrity and Ethics: New Regulation in Scandinavia

Researchers sometimes engage in various forms of dishonesty and unethical behavior, which has led to regulatory efforts to ensure that they work according to acceptable standards. Such regulation is a difficult task, as research is a diverse and dynamic endeavor. Researchers can disagree about what counts as good and acceptable standards, and these standards are constantly developing. This paper presents and discusses recent changes in research integrity and ethics regulation in Norway, Denmark, and Sweden. Recognizing that research norms are developed through practice and are therefore unsuited for comprehensive national regulation, the Scandinavian countries focus on empowering the research community to regulate itself instead, except for the most severe cases of misconduct. This empowerment takes the form of giving research institutions tools and investigatory powers while also holding them responsible for ensuring that both the institution and individual researchers are up to date on relevant norms. In this way, the Scandinavian governments seek to avoid some of the challenges found in more legalistic approaches, which risk lagging behind the continuous development of research norms and can be insensitive to the fact that different disciplines have different norms. While the new approach in Scandinavian has several potential benefits, it also involves potential trade-offs and limitations. The new laws can create confusion about what researchers are allowed to do. Another issue is that it only addresses the fundamental drivers of misconduct to a limited extent.

Stakeholders' Experiences of Research Integrity Support in Universities: A Qualitative Study in Three European Countries

Fostering research integrity (RI) increasingly focuses on normative guidance and supportive measures within institutions. To be successful, the implementation of support should be informed by stakeholders' experiences of RI support. This study aims to explore experiences of RI support in Dutch, Spanish and Croatian universities. In total, 59 stakeholders (Netherlands n = 25, Spain n = 17, Croatia n = 17) participated in 16 focus groups in three European countries. Global themes on RI support experiences were identified by thematic analysis. Themes identified were: 'RI governance and institutional implementation', 'RI roles and structures', 'RI education and supervision', and 'Infrastructure, technology and tools supporting daily practice'. Experiences of support differed between countries in relation to: the efforts to translate norms into practice; the extent to which RI oversight was a responsibility of RE structures, or separate RI structures; and the availability of support close to research practice, such as training, responsible supervision, and adequate tools and infrastructure. The study reinforces the importance of a whole institutional approach to RI, embedded within local jurisdictions, rules, and practices. A whole institutional approach puts the emphasis of responsibility on institutions rather than individual researchers. When such an approach is lacking, some stakeholders look for intervention by authorities, such as funders, outside of the university.

Citation of retracted research: a case-controlled, ten-year follow-up scientometric analysis of Scott S. Reuben’s malpractice

A major problem in scientific literature is the citation of retracted research. Until now, no long-term follow-up of the course of citations of such articles has been published. In the present study, we determined the development of citations of retracted articles based on the case of anaesthesiologist and pain researcher Scott S. Reuben, over a period of 10 years and compared them to matched controls. We screened four databases to find retracted publications by Scott S. Ruben and reviewed full publications for indications of retraction status. To obtain a case-controlled analysis, all Reuben’s retracted articles were compared with the respective citations of the preceeding and subsequent neighbouring articles within the same journal. There were 420 citations between 2009 and 2019, of which only 40% indicated the publication being retracted. Over a 10-year period, an increasing linear trend is observed in citations of retracted articles by Scott S. Ruben that are not reported as retracted (R2 = 0.3647). Reuben’s retracted articles were cited 92% more often than the neighbouring non-retracted articles. This study highlights a major scientific problem. Invented or falsified data are still being cited after more than a decade, leading to a distortion of the evidence and scientometric parameters.

RESPONSE_ABILITY A Card-Based Engagement Method to Support Researchers' Ability to Respond to Integrity Issues

Issues related to research integrity receive increasing attention in policy discourse and beyond with most universities having introduced by now courses addressing issues of good scientific practice. While communicating expectations and regulations related to good scientific practice is essential, criticism has been raised that integrity courses do not sufficiently address discipline and career-stage specific dimensions, and often do not open up spaces for in-depth engagement. In this article, we present the card-based engagement method RESPONSE_ABILITY, which aims at supporting researchers in developing their ability to respond to challenges of good scientific practice. The method acknowledges that what counts and what does not count as acceptable practice may not be as clear-cut as imagined and that research environments matter when it comes to integrity issues. Using four sets of cards as stimulus material, participants are invited to reflect individually and collectively about questions of research integrity from different perspectives. This approach is meant to train them to negotiate in which contexts certain practices can still be regarded as acceptable and where possible transgressions might begin. RESPONSE_ABILITY can be seen as fostering the creation of an integrity culture as it invites a more reflexive engagement with ideals and realities of good practice and opens a space to address underlying value conflicts researchers may be confronted with. Concluding the article, we call for caution that addressing issues of integrity meaningfully requires striking a delicate balance between raising researchers' awareness of individual responsibilities and creating institutional environments that allow them to be response-able.

Research Integrity and Hidden Value Conflicts

Research integrity is a well-established term used to talk and write about ethical issues in research. Part of its success might be its broad applicability. In this paper, we suggest that this might also be its Achilles heel, since it has the potential to conceal important value conflicts. We identify three broad domains upon which research integrity is applied in the literature: (1) the researcher (or research group), (2) research, and (3) research-related institutions and systems. Integrity in relation to researchers concerns character, although it remains to specify precisely what character traits are the desirable ones in this context and what values researchers should endorse. Integrity in relation to research concerns correct and sufficient description of the research process, data, results, and overall ‘research record’. Hence, it concerns the quality of research. However, whether or not this notion of research integrity covers all ethical aspects of research depends on whether one endorses a wider or a narrower interpretation of the ‘research process’. Integrity in relation to research-related institutions and systems leaves open whether they should be understood as agents in their own right or merely as means to research integrity. Besides the potential lack of clarity that our analysis reveals, we point to how this variety in uses might lead to concealment of value conflicts and propose an open discussion of central values.

Research Ethical Norms, Guidance and the Internet

The internet, either as a tool or as an area of research, adds moral worries to an already complicated research ethical backdrop. Agencies, professional associations and philosophers have formulated research ethical norms designed to help scientists to arrive at responsible solutions to the problems. Yet, many criticize this reliance on norms. Somewhat more precisely, many claim that research ethical norms do not offer guidance. In the literature at least three arguments to that effect can be found. First, the research ethical norms fail to guide since they are inconsistent. Second, they fail to guide since they are too opaque. Third, they fail to guide since they cannot handle the moral complexity of issues scientists doing e-research face. In this paper I argue that these arguments are weak. The arguments are, in their original formulations, rather unclear. I try to improve the situation by spelling out the arguments with reference to a certain set of research ethical norms, to a certain account of action-guidance and with reference to certain important distinctions. It then turns out that the arguments' premises are either untenable or suffers from lack of relevance. The arguments do not force us to conclude that research ethical norms fail to guide.

Responsible research and innovation training programs: implementation and evaluation of the HEIRRI project

Responsible research and innovation, or RRI, is a concept that aims to bring together society and science for a better future. There are six key elements of RRI: public engagement, gender equality, science education, open access, ethics and governance.

Higher Education Institutions and Responsible Research and Innovation (HEIRRI) project aimed to bring the concept of RRI into the educational system. Using state-of-the-art review of good practices, HEIRRI team developed 10 training programs on RRI for different higher education institution educational levels, including a summer school and a massive open online course (MOOC). We conducted pilot of the trainings and evaluated participants’ experiences. Satisfaction with HEIRRI training programs on responsible research and innovation was high, both for participants and for the trainers, and trainings raised awareness of RRI. Participants’ feedback was used to identify areas that need improvement and provided for recommendations for final versions of the HEIRRI training programs.

In order to equip researchers with skills to recognize and apply RRI values, RRI should be included in their education. HEIRRI training is suitable for a range of different disciplines, including forensic science, and is free to use and adjust for specific contexts (available from: https://rri-tools.eu/heirri-training-programmes).

Supplemental data for this article is available online at https://doi.org/10.1080/20961790.2021.1970319 .

Development and implementation of research integrity guidance documents: Explorative interviews with research integrity experts

Research integrity (RI) guidance documents often lack sufficient details on handling specific RI issues causing the lack of harmonized approaches to RI and opening the way to research misconduct and other detrimental research practices. Standard operating procedures (SOPs) are developed and implemented by organizations for ensuring the uniformity and quality of performed actions. This study aimed to explore stakeholders' opinions on SOPs for RI, factors influencing the implementation of RI guidance documents and practices, and ideas for improvements in the RI field. We conducted semi-structured interviews with stakeholders from different groups. Data were analyzed using the reflexive thematic analysis approach, and three themes were developed. The first theme addressed participants' knowledge and perceptions on SOPs for RI and their impact on RI promotion and implementation. The second theme described different factors that have a positive or negative impact on the implementation of RI and RI guidance documents and practices, while the third theme addressed needed changes and ideas for improvements in the RI field. Participants considered SOPs valuable for RI promotion. SOPs should be developed based on and consistent with more general and aspirational guidance and through the dialogue with researchers and other stakeholders, to ensure their relevancy.

Exploring the Gray Area: Similarities and Differences in Questionable Research Practices (QRPs) Across Main Areas of Research

This paper explores the gray area of questionable research practices (QRPs) between responsible conduct of research and severe research misconduct in the form of fabrication, falsification, and plagiarism (Steneck in SEE 12(1): 53-57, 2006). Up until now, we have had very little knowledge of disciplinary similarities and differences in QRPs. The paper is the first systematic account of variances and similarities. It reports on the findings of a comprehensive study comprising 22 focus groups on practices and perceptions of QRPs across main areas of research. The paper supports the relevance of the idea of epistemic cultures (Knorr Cetina in: Epistemic cultures: how the sciences make knowledge, Harvard University Press, Cambridge, 1999), also when it comes to QRPs. It shows which QRPs researchers from different areas of research (humanities, social sciences, medical sciences, natural sciences, and technical sciences) report as the most severe and prevalent within their fields. Furthermore, it shows where in the research process these self-reported QRPs can be found. This is done by using a five-phase analytical model of the research process (idea generation, research design, data collection, data analysis, scientific publication and reporting). The paper shows that QRPs are closely connected to the distinct research practices within the different areas of research. Many QRPs can therefore only be found within one area of research, and QRPs that cut across main areas often cover relatively different practices. In a few cases, QRPs in one area are considered good research practice in another.

Towards a Research Agenda for Promoting Responsible Research Practices

This opinion piece aims to inform future research funding programs on responsible research practices (RRP) based on three specific objectives: (1) to give a sketch of the current international discussion on responsible research practices (RRPs); (2) to give an overview of current initiatives and already obtained results regarding RRP; and (3) to give an overview of potential future needs for research on RRP. In this opinion piece, we have used seven iterative methodological steps (including literature review, ranking, and sorting exercises) to create the proposed research agenda. We identified six main themes that we believe need attention in future research: (1) responsible evaluation of research and researchers, (2) the influence of open science and transparency on RRP, (3) research on responsible mentoring, supervision, and role modeling, (4) the effect of education and training on RRP, (5) checking for reproducibility, and (6) responsible and fair peer review. These themes have in common that they address aspects of research that are mostly on the level of the scientific system, more than on the level of the individual researcher. Some current initiatives are already gathering substantial empirical evidence to start filling these gaps. We believe that with sufficient support from all relevant stakeholders, more progress can be made.

Atmospheres of science: Experiencing scientific mobility

This article uses notions of the atmospheric to engage with empirical material concerned with international mobility in science. It draws on recent conceptual work on atmospheres that frames them as allowing access to the affective qualities of everyday life and as 'productively nebulous': atmospheres exist between the local and the globally diffuse, the emergent and the staged, the intangible and the brutally present. Using atmospheric thinking, I devise 'apparatuses of attunement' to capture elusive aspects of life in science, as discussed in interviews with natural scientists about their experiences of international mobility. In particular I use ideas of the situation, atmospheric threads, and the staging of atmospheres to argue that scientists represent themselves as existing in between the particular and the general: they are never wholly at the mercy of the structures and expectations of globalized science, but are also never not in the grip of them. In closing I reflect on what this analysis reveals about the affective qualities of contemporary science, the forms of life being nurtured by the norms and expectations of research (policy), and the kinds of agency available to (these) scientists. The aim of the article is thus twofold: to demonstrate how concepts of atmospheres can be put to work in STS, and to contribute to research on international mobility in science and contemporary scientific careers.

Expanding Research Integrity: A Cultural-Practice Perspective

Research integrity (RI) is usually discussed in terms of responsibilities that individual researchers bear towards the scientific work they conduct, as well as responsibilities that institutions have to enable those individual researchers to do so. In addition to these two bearers of responsibility, a third category often surfaces, which is variably referred to as culture and practice. These notions merit further development beyond a residual category that is to contain everything that is not covered by attributions to individuals and institutions. This paper discusses how thinking in RI can take benefit from more specific ideas on practice and culture. We start by articulating elements of practice and culture, and explore how values central to RI are related to these elements. These insights help identify additional points of intervention for fostering responsible conduct. This helps to build "cultures and practices of research integrity", as it makes clear that specific times and places are connected to specific practices and cultures and should have a place in the debate on Research Integrity. With this conceptual framework, practitioners as well as theorists can avoid using the notions as residual categories that de facto amount to vague, additional burdens of responsibility for the individual.

Rethinking success, integrity, and culture in research (part 1) - a multi-actor qualitative study on success in science

BACKGROUND

Success shapes the lives and careers of scientists. But success in science is difficult to define, let alone to translate in indicators that can be used for assessment. In the past few years, several groups expressed their dissatisfaction with the indicators currently used for assessing researchers. But given the lack of agreement on what should constitute success in science, most propositions remain unanswered. This paper aims to complement our understanding of success in science and to document areas of tension and conflict in research assessments.

METHODS

We conducted semi-structured interviews and focus groups with policy makers, funders, institution leaders, editors or publishers, research integrity office members, research integrity community members, laboratory technicians, researchers, research students, and former-researchers who changed career to inquire on the topics of success, integrity, and responsibilities in science. We used the Flemish biomedical landscape as a baseline to be able to grasp the views of interacting and complementary actors in a system setting.

RESULTS

Given the breadth of our results, we divided our findings in a two-paper series, with the current paper focusing on what defines and determines success in science. Respondents depicted success as a multi-factorial, context-dependent, and mutable construct. Success appeared to be an interaction between characteristics from the researcher (Who), research outputs (What), processes (How), and luck. Interviewees noted that current research assessments overvalued outputs but largely ignored the processes deemed essential for research quality and integrity. Interviewees suggested that science needs a diversity of indicators that are transparent, robust, and valid, and that also allow a balanced and diverse view of success; that assessment of scientists should not blindly depend on metrics but also value human input; and that quality should be valued over quantity.

CONCLUSIONS

The objective of research assessments may be to encourage good researchers, to benefit society, or simply to advance science. Yet we show that current assessments fall short on each of these objectives. Open and transparent inter-actor dialogue is needed to understand what research assessments aim for and how they can best achieve their objective.

STUDY REGISTRATION

osf.io/33v3m.

Rethinking success, integrity, and culture in research (part 2) - a multi-actor qualitative study on problems of science

BACKGROUND

Research misconduct and questionable research practices have been the subject of increasing attention in the past few years. But despite the rich body of research available, few empirical works also include the perspectives of non-researcher stakeholders.

METHODS

We conducted semi-structured interviews and focus groups with policy makers, funders, institution leaders, editors or publishers, research integrity office members, research integrity community members, laboratory technicians, researchers, research students, and former-researchers who changed career to inquire on the topics of success, integrity, and responsibilities in science. We used the Flemish biomedical landscape as a baseline to be able to grasp the views of interacting and complementary actors in a system setting.

RESULTS

Given the breadth of our results, we divided our findings in a two-paper series with the current paper focusing on the problems that affect the integrity and research culture. We first found that different actors have different perspectives on the problems that affect the integrity and culture of research. Problems were either linked to personalities and attitudes, or to the climates in which researchers operate. Elements that were described as essential for success (in the associate paper) were often thought to accentuate the problems of research climates by disrupting research culture and research integrity. Even though all participants agreed that current research climates need to be addressed, participants generally did not feel responsible nor capable of initiating change. Instead, respondents revealed a circle of blame and mistrust between actor groups.

CONCLUSIONS

Our findings resonate with recent debates, and extrapolate a few action points which might help advance the discussion. First, the research integrity debate must revisit and tackle the way in which researchers are assessed. Second, approaches to promote better science need to address the impact that research climates have on research integrity and research culture rather than to capitalize on individual researchers' compliance. Finally, inter-actor dialogues and shared decision making must be given priority to ensure that the perspectives of the full research system are captured. Understanding the relations and interdependency between these perspectives is key to be able to address the problems of science.

STUDY REGISTRATION

https://osf.io/33v3m.