A qualitative approach to Responsible Conduct of Research (RCR) training development: identification of metacognitive strategies

Research ethics courses as a vaccination against a toxic research environment or culture

Hofmann and Holm’s (2019) recent survey on issues of research misconduct with PhD graduates culminated with a notable conclusion by the authors: ‘ Scientific misconduct seems to be an environmental issue as much as a matter of personal integrity’. Here, we re-emphasise the usefulness of an education-based countermeasure against toxic research environments or cultures that promote unethical practices amongst the younger researchers. We posit that an adequately conducted course in research ethics and integrity, with a good dose of case studies and analyses, can function in a manner that is metaphorically akin to vaccination. The training would cultivate the ability to analyse and build confidence in young researchers in making decisions with sound moral reasoning as well as in speaking up or arguing against pressure and coercions into unacceptable behaviour. A sufficiently large number of young researchers exposed to research ethics trainings would essentially provide a research community some degree of lasting herd immunity at its broadest base. Beyond passive immunity, a crop of research ethics-savvy young researchers could also play active and influential roles as role models for others at their level and perhaps even help correct the wayward attitudes of some senior researchers and initiate prompt action from institutional policy makers in a bottom-up manner.

A Reflective Account of a Research Ethics Course for an Interdisciplinary Cohort of Graduate Students

The graduate course in research ethics in the Graduate School for Integrative Sciences and Engineering at the National University of Singapore consists of a semester long mandatory course titled: "Research Ethics and Scientific Integrity." The course provides students with guiding principles for appropriate conduct in the professional and social settings of scientific research and in making morally weighted and ethically sound decisions when confronted with moral dilemmas. It seeks to enhance understanding and appreciation of the moral reasoning underpinning various rules and legislative constraints associated with research subjects and procedures. Further, students are trained to critically analyse cases and issues associated with scientific misconduct preparing them to act in a responsible and effective manner should they encounter such cases. The diverse background and training of the cohort also provide a unique setting and opportunity for student-initiated collaborative interdisciplinary learning. This article offers a reflective account of the course and some preliminary insights into learning outcomes.

Clinical translational students' perceptions of research ethics coursework: a case study

Relatively unknown is whether coursework in responsible conduct of research actually achieve the purposes for which it is designed. In this study, the authors report clinical translational students’ perceptions of their research ethics coursework and the alignment between course content as recommended in the literature. We used grounded theory to portray emergent findings across focus groups and semistructured interviews among 31 participants at one clinical translational science hub. We also used thematic analysis to analyze course syllabi. Two themes emerged: Averting scientific misconduct and Responding to ethical dilemmas. Students reported that they did not acquire requisite strategies to address research ethical dilemmas. One of the course syllabi indicated the provision of active learning opportunities. However, the findings did not offer support. Developing experiential learning activities and ensuring that course content is aligned with the contemporary ethical practices, such as case study and portfolio development, is recommended.

To Whistleblow or Not to Whistleblow: Affective and Cognitive Differences in Reporting Peers and Advisors

Traditional whistleblowing theories have purported that whistleblowers engage in a rational process in determining whether or not to blow the whistle on misconduct. However, stressors inherent to whistleblowing often impede rational thinking and act as a barrier to effective whistleblowing. The negative impact of these stressors on whistleblowing may be made worse depending on who engages in the misconduct: a peer or advisor. In the present study, participants are presented with an ethical scenario where either a peer or advisor engages in misconduct, and positive and the negative consequences of whistleblowing are either directed to the wrongdoer, department, or university. Participant responses to case questions were evaluated for whistleblowing intentions, moral intensity, metacognitive reasoning strategies, and positive and negative, active and passive emotions. Findings indicate that participants were less likely to report the observed misconduct of an advisor compared to a peer. Furthermore, the findings also suggest that when an advisor is the source of misconduct, greater negative affect results. Post-hoc analyses were also conducted examining the differences between those who did and did not intend to blow the whistle under the circumstances of either having to report an advisor or peer. The implications of these findings for understanding the complexities involved in whistleblowing are discussed.

Qualitative Evaluation Methods in Ethics Education: A Systematic Review and Analysis of Best Practices

Although qualitative research offers some unique advantages over quantitative research, qualitative methods are rarely employed in the evaluation of ethics education programs and are often criticized for a lack of rigor. This systematic review investigated the use of qualitative methods in studies of ethics education. Following a review of the literature in which 24 studies were identified, each study was coded based on 16 best practices characteristics in qualitative research. General thematic analysis and grounded theory were found to be the dominant approaches used. Researchers are effectively executing a number of best practices, such as using direct data sources, structured data collection instruments, non-leading questioning, and expert raters. However, other best practices were rarely present in the courses reviewed, such as collecting data using multiple sources, methods, raters, and timepoints, evaluating reliability, and employing triangulation analyses to assess convergence. Recommendations are presented for improving future qualitative research studies in ethics education.

Scientific Misconduct

Scientific misconduct has been defined as fabrication, falsification, and plagiarism. Scientific misconduct has occurred throughout the history of science. The US government began to take systematic interest in such misconduct in the 1980s. Since then, a number of studies have examined how frequently individual scientists have observed scientific misconduct or were involved in it. Although the studies vary considerably in their methodology and in the nature and size of their samples, in most studies at least 10% of the scientists sampled reported having observed scientific misconduct. In addition to studies of the incidence of scientific misconduct, this review considers the recent increase in paper retractions, the role of social media in scientific ethics, several instructional examples of egregious scientific misconduct, and potential methods to reduce research misconduct.

Team-based learning instruction for responsible conduct of research positively impacts ethical decision-making

Common practices for responsible conduct of research (RCR) instruction have recently been shown to have no positive impact on and possibly to undermine ethical decision-making (EDM). We show that a team-based learning (TBL) RCR curriculum results in some gains in decision ethicality, the use of more helpful metacognitive reasoning strategies in decision-making, and elimination of most negative effects of other forms of RCR instruction on social-behavioral responses. TBL supports the reasoning strategies and social mechanisms that underlie EDM and ethics instruction, and may provide a more effective method for RCR instruction than lectures and small group discussion.

Competition and Sensemaking in Ethical Situations

Intra-organizational competition was examined in relation to ethicality. The effect of a competitor being an in-group versus and out-group member, competitor offering uncorroborated or corroborated information, and the impact of the competitor expressing selfish, pro-group, or pro-organizational level goals were examined. Findings suggest that the way competition is presented has an important influence on how well individuals are able to make sense of an ethically ambiguous situation and render an ethical decision. A main effect for information sharing was found, such that when a competitor offers uncorroborated information participants made less ethical decisions and used pro-ethical reasoning strategies less often. An additional main effect was found suggesting that participants made more ethical decisions when working with an in-group competitor rather than an out-group competitor. Complex interactive effects were also found and discussed suggesting that pro-ethical reasoning strategies may be used less often depending on information corroboration, the competitor's relative group membership status, and the motives expressed by the competitor.

Assessing graduate student progress in engineering ethics

Under a grant from the National Science Foundation, the authors (and others) undertook to integrate ethics into graduate engineering classes at three universities-and to assess success in a way allowing comparison across classes (and institutions). This paper describes the attempt to carry out that assessment. Standard methods of assessment turned out to demand too much class time. Under pressure from instructors, the authors developed an alternative method that is both specific in content to individual classes and allows comparison across classes. Results are statistically significant for ethical sensitivity and knowledge. They show measurable improvement in a single semester.

Applying Cases to Solve Ethical Problems: The Significance of Positive and Process-Oriented Reflection

This study examined the role of reflection on personal cases for making ethical decisions with regard to new ethical problems. Participants assumed the position of a business manager in a hypothetical organization and solved ethical problems that might be encountered. Prior to making a decision for the business problems, participants reflected on a relevant ethical experience. The findings revealed that application of material garnered from reflection on a personal experience was associated with decisions of higher ethicality. However, whether the case was viewed as positive or negative, and whether the outcomes, process, or outcomes and processes embedded in the experience were examined, influenced the application of case material to the new problem. As expected, examining positive experiences and the processes involved in those positive experiences resulted in greater application of case material to new problems. Future directions and implications for understanding ethical decision-making are discussed.

Intrinsic ethics regarding integrated assessment models for climate management

In this essay we develop and argue for the adoption of a more comprehensive model of research ethics than is included within current conceptions of responsible conduct of research (RCR). We argue that our model, which we label the ethical dimensions of scientific research (EDSR), is a more comprehensive approach to encouraging ethically responsible scientific research compared to the currently typically adopted approach in RCR training. This essay focuses on developing a pedagogical approach that enables scientists to better understand and appreciate one important component of this model, what we call intrinsic ethics. Intrinsic ethical issues arise when values and ethical assumptions are embedded within scientific findings and analytical methods. Through a close examination of a case study and its application in teaching, namely, evaluation of climate change integrated assessment models, this paper develops a method and case for including intrinsic ethics within research ethics training to provide scientists with a comprehensive understanding and appreciation of the critical role of values and ethical choices in the production of research outcomes.

Outcomes assessment of role-play scenarios for teaching responsible conduct of research

We describe the summative assessment of role-play scenarios that we previously developed to teach central topics in the responsible conduct of research (RCR) to graduate students in science and engineering. Interviews with role-play participants, with participants in a case discussion training session, and with untrained students suggested that role-playing might promote a deeper appreciation of RCR by shifting the focus away from wanting to simply "know the rules." We also present the results of a think-aloud case analysis study and describe the development of a behaviorally-anchored rating scale (BARS) to assess participants' case analysis performance.

The Influence of Temporal Orientation and Affective Frame on use of Ethical Decision-Making Strategies

This study examined the role of temporal orientation and affective frame in the execution of ethical decision-making strategies. In reflecting on a past experience or imagining a future experience, participants thought about experiences that they considered either positive or negative. The participants recorded their thinking about that experience by responding to several questions, and their responses were content-analyzed for the use of ethical decision-making strategies. The findings indicated that a future temporal orientation was associated with greater strategy use. Likewise, a positive affective frame was associated with greater strategy use. Future orientation may permit better strategy execution than a past orientation because it facilitates more objective, balanced contemplation of the reflected-upon situation, and minimizes potential self-threat associated with past behavior. A positive affective frame likely improves strategy execution because it facilitates active analysis of the experience. Future directions and implications of these findings are discussed.

Instruction in the responsible conduct of research: an inventory of programs and materials within CTSAs

The National Institutes of Health (NIH) require instruction in the responsible conduct of research (RCR) as a component of any Clinical and Translational Science Award (CTSA). The Educational Materials Group of the NIH CTSA Consortium's Clinical Research Ethics Key Function Committee (CRE-KFC) conducted a survey of the 38 institutions that held CTSA funding as of January 2009 to determine how they satisfy RCR training requirements. An 8-item questionnaire was sent by email to directors of the Clinical Research Ethics, the Educational and Career Development, and the Regulatory Knowledge cores. We received 78 completed surveys from 38 CTSAs (100%). We found that there is no unified approach to RCR training across CTSAs, many programs lack a coherent plan for RCR instruction, and most CTSAs have not developed unique instructional materials tailored to the needs of clinical and translational scientists. We recommend collaboration among CTSAs and across CTSA key function committees to address these weaknesses. We also requested that institutions send electronic copies of original RCR training materials to share among CTSAs via the CTSpedia website. Twenty institutions submitted at least one educational product. The CTSpedia now contains more than 90 RCR resources.

Development of role-play scenarios for teaching responsible conduct of research

We describe the development, testing, and formative evaluation of nine role-play scenarios for teaching central topics in the responsible conduct of research to graduate students in science and engineering. In response to formative evaluation surveys, students reported that the role-plays were more engaging and promoted deeper understanding than a lecture or case study covering the same topic. In the future, summative evaluations will test whether students display this deeper understanding and retain the lessons of the role-play experience.

Evaluating the effects that existing instruction on responsible conduct of research has on ethical decision making

PURPOSE

To examine the effects that existing courses on the responsible conduct of research (RCR) have on ethical decision making by assessing the ethicality of decisions made in response to ethical problems and the underlying processes involved in ethical decision making. These processes included how an individual thinks through ethical problems (i.e., meta-cognitive reasoning strategies) and the emphasis placed on social dimensions of ethical problems (i.e., social-behavioral responses).

METHOD

In 2005-2007, recruitment announcements were made, stating that a nationwide, online study was being conducted to examine the impact of RCR instruction on the ethical decision making of scientists. Recruitment yielded contacts with over 200 RCR faculty at 21 research universities and medical schools; 40 (20%) RCR instructors enrolled their courses in the current study. From those courses, 173 participants completed an ethical decision-making measure.

RESULTS

A mixed pattern of effects emerged. The ethicality of decisions did not improve as a result of RCR instruction and even decreased for decisions pertaining to business aspects of research, such as contract bidding. Course participants improved on some meta-cognitive reasoning strategies, such as awareness of the situation and consideration of personal motivations, but declined for seeking help and considering others' perspectives. Participants also increased their endorsement of detrimental social-behavioral responses, such as deception, retaliation, and avoidance of personal responsibility.

CONCLUSIONS

These findings indicated that RCR instruction may not be as effective as intended and, in fact, may even be harmful. Harmful effects might result if instruction leads students to overstress avoidance of ethical problems, be overconfident in their ability to handle ethical problems, or overemphasize their ethical nature. Future research must examine these and other possible obstacles to effective RCR instruction.

Creativity and Ethics: The Relationship of Creative and Ethical Problem-Solving

Students of creativity have long been interested in the relationship between creativity and deviant behaviors such as criminality, mental disease, and unethical behavior. In the present study we wished to examine the relationship between creative thinking skills and ethical decision-making among scientists. Accordingly, 258 doctoral students in the health, biological, and social sciences were asked to complete a measure of creative processing skills (e.g., problem definition, conceptual combination, idea generation) and a measure of ethical decision-making examining four domains, data management, study conduct, professional practices, and business practices. It was found that ethical decision-making in all four of these areas was related to creative problem-solving processes with late cycle processes (e.g., idea generation and solution monitoring) proving particularly important. The implications of these findings for understanding the relationship between creative and deviant thought are discussed.

Discourses of Dislike: Responses to Ethics Education Policies by Life Scientists in the U.K., Italy, and the U.S

Recently established policies for ethical conduct of researchers have resulted in efforts to implement those policies through educational practices. While these policies and training efforts have good intentions, how do researchers respond? Little research has been conducted to understand how researchers react to ethics policies, and to ethics education requirements as a particularly salient part of the policies. This research explores variations in the responses of life scientists to policies originating in the U.K., E.U. and U.S., between 2000 and 2003. Semi-structured interviews with 30 life scientists in the U.K., Italy and the U.S. provide the basis for the findings. Across the interviews, scientists consistently reported dislike of the ways the ethics policies and ethics programs are implemented, but with variation by country. U.K. life scientists regarded the policies tied to Research Council (RC) funding with a skeptical attitude, expecting that policies would change with RC leadership. Italian life scientists often noted the lack of funding (and policies) at the national level, and appeared frustrated by what they viewed as overly complex policy requirements at the European level, where successful grantees employed outside consultants to complete the ethics sections of their applications. U.S. life scientists expressed distaste for the implementation of the National Institutes of Health (NIH) policy in web-based ethics training modules. This research suggests that introducing new shallow routines that are separate from daily research and educational practices in science may be worse than no ethics training at all, if it turns off young researchers to the discussion of ethical issues. Observing variation in scientists' discourse on ethics policies allows us to see how education following ethics policies might be more effective, and indicates that ethics policy formation more informed by empirical research is a desirable goal.

Mental models: an alternative evaluation of a sensemaking approach to ethics instruction

In spite of the wide variety of approaches to ethics training it is still debatable which approach has the highest potential to enhance professionals' integrity. The current effort assesses a novel curriculum that focuses on metacognitive reasoning strategies researchers use when making sense of day-to-day professional practices that have ethical implications. The evaluated trainings effectiveness was assessed by examining five key sensemaking processes, such as framing, emotion regulation, forecasting, self-reflection, and information integration that experts and novices apply in ethical decision-making. Mental models of trained and untrained graduate students, as well as faculty, working in the field of physical sciences were compared using a think-aloud protocol 6 months following the ethics training. Evaluation and comparison of the mental models of participants provided further validation evidence for sensemaking training. Specifically, it was found that trained students applied metacognitive reasoning strategies learned during training in their ethical decision-making that resulted in complex mental models focused on the objective assessment of the situation. Mental models of faculty and untrained students were externally-driven with a heavy focus on autobiographical processes. The study shows that sensemaking training has a potential to induce shifts in researchers' mental models by making them more cognitively complex via the use of metacognitive reasoning strategies. Furthermore, field experts may benefit from sensemaking training to improve their ethical decision-making framework in highly complex, novel, and ambiguous situations.