Initial Results from the Survey of Organizational Research Climates (SOuRCe) in the U.S. Department of Veterans Affairs Healthcare System

Perceptions of the Research Integrity Climate in Egyptian Universities: A Survey Among Academic Researchers

Problem: Investigations regarding perceptions of the institutional research integrity climate in the Arab Middle East remain underexplored. Subjects: We surveyed faculty from three Egyptian universities. Method: We utilized the Survey of Organizational Research Climate (SOuRCe) tool, which incorporates seven subscales that measure different aspects of the research integrity climate. Responses were obtained from a 5-point Likert scale. Findings: Of the 228 participants, the subscales 'Regulatory Quality' and '[Lack of] Integrity Inhibitors' received the highest mean scores, whereas the lowest scores pertained to 'Departmental Expectations,' 'Integrity Socialization,' and 'Responsible Conduct of Research´ indicating areas in need of improvement. Conclusions: Academic leaders should set fairer expectations for research and funding for their researchers, ensure junior researchers are socialized into research integrity practices, and promote effective RCR training and availability of RCR policies. We identify specific targeted interventions to enhance the research integrity climate within these institutions.

Perceptions of Research Integrity Climate in Hungarian Universities: Results from A Survey among Academic Researchers

Research integrity climate is an important factor that influences an individual's behavior. A strong research integrity culture can lead to better research practices and responsible conduct of research (RCR). Therefore, investigations on organizational climate can be a valuable tool to identify the strengths and weaknesses of each group and develop targeted initiatives. This study aims to assess the perceptions on integrity climate in three universities in Hungary. A cross-sectional study was conducted with PhD students, postdocs, and professors from three Hungarian universities. The survey included demographic questions, such as gender, age, scientific field, academic rank, and the Survey of Organizational Research Climate (SOURCE). A total of 432 participants completed the survey. Our results show that postdocs and assistant professors perceived integrity climate more negatively than PhD students and full professors in every survey scale. Contrarily, PhD students perceive more positively than the other groups. Disciplinary differences show that researchers in the Biomedical sciences perceive regulatory bodies to be fairer when evaluating their projects than those in the Natural sciences. Natural sciences also perceive more negatively how the department values integrity when compared to Humanities. Humanities perceive more positively Advisor/Advisee Relations than Biomedical Sciences. Our results suggest that institutions should pay more attention to early career researchers, especially insecure and temporary positions like postdocs and assistant professors. They should provide RCR resources, socialize them in RCR, and set more reasonable expectations. Moreover, department leaders should develop initiatives to foster better integrity climates.

Reliability and validation of an attitude scale regarding responsible conduct in research

Background

Several studies reveal a problematic prevalence of research misbehaviors. There are several potential causes of research misconduct but ensuring that scientists hold attitudes that reflect norms of acceptable behaviors is fundamental.

Aim

Our aim was to evaluate the psychometric properties (factor structure and reliability) of an “attitude” scale that we adopted from a questionnaire we previously used to investigate the prevalence of research misbehaviors in the Middle East.

Methods

We used data from participants (n = 254) who were involved in our prior questionnaire study to determine the validity of an attitude scale that we adapted from this previous study. We performed exploratory factor analysis (EFA) to determine the factor structure of the attitude scale followed by measures of convergent and concurrent validity. We assessed reliability by computing the Cronbach’s alphas of each construct of the attitude scale.

Results

EFA indicated that the attitude scale consists of two factors (constructs). Convergent validity was demonstrated by significant correlations of item-item and item-total. Correlation analysis revealed that the attitude constructs were significantly correlated with the Research Misbehavior Severity Score, thereby demonstrating concurrent validity. Cronbach’s alphas were greater than 0.75 for both constructs.

Conclusion

We demonstrated a valid and reliable 20-item attitude scale with two factors related to “acceptability of practices in responsible conduct in research” and “general attitudes regarding scientific misconduct”. The use of a validated attitude scale can help assess the effectiveness of educational programs that focus on participants acquiring attitudes that are instrumental in responsible conduct in research.

Assessing the climate for research ethics in labs: Development and validation of a brief measure

The environment researchers work in influences their ethical decisions and behavior. A "climate" for research ethics in a research lab exists when members of the lab perceive that the group values and is committed to principles of research ethics. In this study, we aimed to develop a short, reliable and valid measure assessing perceptions of climate for research ethics at the lab level. The resulting measure, Lab Climate for Research Ethics, was developed using standard scale development guidelines. In a large sample of postdoctoral researchers (N = 570), we found preliminary evidence that the new measure has adequate internal consistency reliability. It was also correlated with an existing measure of climate for research ethics and was not correlated with social desirability, demonstrating evidence of construct validity. The new measure can be used in a variety of contexts, including research administrators seeking information about climate within labs across an institution and researchers who study lab environments.

Reflexive Behaviour: How Publication Pressure Affects Research Quality in Astronomy

Reflexive metrics is a branch of science studies that explores how the demand for accountability and performance measurement in science has shaped the research culture in recent decades. Hypercompetition and publication pressure are part of this neoliberal culture. How do scientists respond to these pressures? Studies on research integrity and organisational culture suggest that people who feel treated unfairly by their institution are more likely to engage in deviant behaviour, such as scientific misconduct. By building up on reflexive metrics, combined with studies on the influence of organisational culture on research integrity, this study reflects on the research behaviour of astronomers with the following questions: (1) To what extent is research (mis-)behaviour reflexive, i.e., dependent on perceptions of publication pressure and distributive and organisational justice? (2) What impact does scientific misconduct have on research quality? In order to perform this reflection, we conducted a comprehensive survey of academic and non-academic astronomers worldwide and received 3509 responses. We found that publication pressure explains 10% of the variance in occurrence of misconduct and between 7% and 13% of the variance of the perception of distributive and organisational justice as well as overcommitment to work. Our results on the perceived impact of scientific misconduct on research quality show that the epistemic harm of questionable research practices should not be underestimated. This suggests there is a need for a policy change. In particular, lesser attention to metrics (such as publication rate) in the allocation of grants, telescope time and institutional rewards would foster better scientific conduct and, hence, research quality.

Expanding Research Integrity: A Cultural-Practice Perspective

Research integrity (RI) is usually discussed in terms of responsibilities that individual researchers bear towards the scientific work they conduct, as well as responsibilities that institutions have to enable those individual researchers to do so. In addition to these two bearers of responsibility, a third category often surfaces, which is variably referred to as culture and practice. These notions merit further development beyond a residual category that is to contain everything that is not covered by attributions to individuals and institutions. This paper discusses how thinking in RI can take benefit from more specific ideas on practice and culture. We start by articulating elements of practice and culture, and explore how values central to RI are related to these elements. These insights help identify additional points of intervention for fostering responsible conduct. This helps to build "cultures and practices of research integrity", as it makes clear that specific times and places are connected to specific practices and cultures and should have a place in the debate on Research Integrity. With this conceptual framework, practitioners as well as theorists can avoid using the notions as residual categories that de facto amount to vague, additional burdens of responsibility for the individual.

Researchers' Perceptions of a Responsible Research Climate: A Multi Focus Group Study

The research climate plays a key role in fostering integrity in research. However, little is known about what constitutes a responsible research climate. We investigated academic researchers' perceptions on this through focus group interviews. We recruited researchers from the Vrije Universiteit Amsterdam and the Amsterdam University Medical Center to participate in focus group discussions that consisted of researchers from similar academic ranks and disciplinary fields. We asked participants to reflect on the characteristics of a responsible research climate, the barriers they perceived and which interventions they thought fruitful to improve the research climate. Discussions were recorded and transcribed at verbatim. We used inductive content analysis to analyse the focus group transcripts. We conducted 12 focus groups with 61 researchers in total. We identified fair evaluation, openness, sufficient time, integrity, trust and freedom to be mentioned as important characteristics of a responsible research climate. Main perceived barriers were lack of support, unfair evaluation policies, normalization of overwork and insufficient supervision of early career researchers. Possible interventions suggested by the participants centered around improving support, discussing expectations and improving the quality of supervision. Some of the elements of a responsible research climate identified by participants are reflected in national and international codes of conduct, such as trust and openness. Although it may seem hard to change the research climate, we believe that the realisation that the research climate is suboptimal should provide the impetus for change informed by researchers' experiences and opinions.

Perceptions of Ethical Climate and Research Pressures in Different Faculties of a University: Cross-Sectional Study at the University of Split, Croatia

We determined the prevailing ethical climate at three different schools of a single university, in order to explore possible differences in the ethical climate related to different research fields: the School of Electrical Engineering, Mechanical Engineering, and Naval Architecture; the School of Humanities and Social Sciences; and the School of Medicine. We used the Ethical Climate Questionnaire to survey the staff (teachers and administration) at the three schools, and used the research integrity and organizational climate (RIOC) survey for early-stage researchers at the three schools. The dominant ethical climate type perceived collectively at the three university schools (response rate 49%, n = 294) was Laws and professional codes, which is associated with the cosmopolitan level of analysis and the ethical construct of principle. Individually, the same climate predominated at the schools for engineering and humanities, but the School of Medicine had the Self-interest ethical climate, which is associated with the individual level of analysis and the egoism ethical construct. In the RIOC survey (response rate 85%; n = 70), early-stage researchers from the three university schools did not differ in their perceptions of the organizational research integrity climate, or in their perceived individual, group or organizational pressures. Our study is the first, to the best of our knowledge, to show differences in perceived ethical climate at a medical school compared to other schools at a university. Further studies are needed to explore the reasons for these differences and how they translate to organizational outcomes, such as job satisfaction, commitment to the institution and dysfunctional behaviour, including research misconduct.

Perceptions of research integrity climate differ between academic ranks and disciplinary fields: Results from a survey among academic researchers in Amsterdam

Breaches of research integrity have shocked the academic community. Initially explanations were sought at the level of individual researchers but over time increased recognition emerged of the important role that the research integrity climate may play in influencing researchers' (mis)behavior. In this study we aim to assess whether researchers from different academic ranks and disciplinary fields experience the research integrity climate differently. We sent an online questionnaire to academic researchers in Amsterdam using the Survey of Organizational Research Climate. Bonferroni corrected mean differences showed that junior researchers (PhD students, postdocs and assistant professors) perceive the research integrity climate more negatively than senior researchers (associate and full professors). Junior researchers note that their supervisors are less committed to talk about key research integrity principles compared to senior researchers (MD = -.39, CI = -.55, -.24). PhD students perceive more competition and suspicion among colleagues (MD = -.19, CI = -.35, -.05) than associate and full professors. We found that researchers from the natural sciences overall express a more positive perception of the research integrity climate. Researchers from social sciences as well as from the humanities perceive less fairness of their departments' expectations in terms of publishing and acquiring funding compared to natural sciences and biomedical sciences (MD = -.44, CI = -.74, -.15; MD = -.36, CI = -.61, -.11). Results suggest that department leaders in the humanities and social sciences should do more to set fairer expectations for their researchers and that senior scientists should ensure junior researchers are socialized into research integrity practices and foster a climate in their group where suspicion among colleagues has no place.

What really matters for successful research environments? A realist synthesis

CONTEXT

Research environments, or cultures, are thought to be the most influential predictors of research productivity. Although several narrative and systematic reviews have begun to identify the characteristics of research-favourable environments, these reviews have ignored the contextual complexities and multiplicity of environmental characteristics.

OBJECTIVES

The current synthesis adopts a realist approach to explore what interventions work for whom and under what circumstances.

METHODS

We conducted a realist synthesis of the international literature in medical education, education and medicine from 1992 to 2016, following five stages: (i) clarifying the scope; (ii) searching for evidence; (iii) assessing quality; (iv) extracting data, and (v) synthesising data.

RESULTS

We identified numerous interventions relating to research strategy, people, income, infrastructure and facilities (IIF), and collaboration. These interventions resulted in positive or negative outcomes depending on the context and mechanisms fired. We identified diverse contexts at the individual and institutional levels, but found that disciplinary contexts were less influential. There were a multiplicity of positive and negative mechanisms, along with three cross-cutting mechanisms that regularly intersected: time; identity, and relationships. Outcomes varied widely and included both positive and negative outcomes across subjective (e.g. researcher identity) and objective (e.g. research quantity and quality) domains.

CONCLUSIONS

The interplay among mechanisms and contexts is central to understanding the outcomes of specific interventions, bringing novel insights to the literature. Researchers, research leaders and research organisations should prioritise the protection of time for research, enculturate researcher identities, and develop collaborative relationships to better foster successful research environments. Future research should further explore the interplay among time, identity and relationships.

Misuse of Coauthorship in Medical Theses in Sweden

The aim of this study was to explore experiences of authorship issues among persons who have recently received their doctoral degree in medicine in Sweden. A survey was mailed to all who received their PhD at a medical faculty at a Swedish university the first half of 2016. Questions concerned experiences of violations of the first three authorship criteria in the Vancouver rules and of misuse of authorship order in the articles of their thesis, and the respondents' attitudes to these matters. The questionnaire was returned by 285 respondents (68%). According to the majority (53%), the Vancouver rules were not fully respected in the articles of their thesis. A vast majority (97%) found it important that authorship issues are handled correctly, but only 19% responded that their department has a clear and consistently applied policy. We conclude that authorship guidelines are frequently disrespected at medical faculties in Sweden. The universities seem to provide limited support on authorship issues.

The Role of Culture and Acculturation in Researchers' Perceptions of Rules in Science

Successfully navigating the norms of a society is a complex task that involves recognizing diverse kinds of rules as well as the relative weight attached to them. In the United States (U.S.), different kinds of rules-federal statutes and regulations, scientific norms, and professional ideals-guide the work of researchers. Penalties for violating these different kinds of rules and norms can range from the displeasure of peers to criminal sanctions. We proposed that it would be more difficult for researchers working in the U.S. who were born in other nations to distinguish the seriousness of violating rules across diverse domains. We administered a new measure, the evaluating rules in science task (ERST), to National Institutes of Health-funded investigators (101 born in the U.S. and 102 born outside of the U.S.). The ERST assessed perceptions of the seriousness of violating research regulations, norms, and ideals, and allowed us to calculate the degree to which researchers distinguished between the seriousness of each rule category. The ERST also assessed researchers' predictions of the seriousness that research integrity officers (RIOs) would assign to the rules. We compared researchers' predictions to the seriousness ratings of 112 RIOs working at U.S. research-intensive universities. U.S.-born researchers were significantly better at distinguishing between the seriousness of violating federal research regulations and violating ideals of science, and they were more accurate in their predictions of the views of RIOs. Acculturation to the U.S. moderated the effects of nationality on accuracy. We discuss the implications of these findings in terms of future research and education.

Engaging multilevel stakeholders in an implementation trial of evidence-based quality improvement in VA women's health primary care

The Veterans Health Administration (VHA) has undertaken primary care transformation based on patient-centered medical home (PCMH) tenets. VHA PCMH models are designed for the predominantly male Veteran population, and require tailoring to meet women Veterans' needs. We used evidence-based quality improvement (EBQI), a stakeholder-driven implementation strategy, in a cluster randomized controlled trial across 12 sites (eight EBQI, four control) that are members of a Practice-Based Research Network. EBQI involves engaging multilevel, inter-professional leaders and staff as stakeholders in reviewing evidence and setting QI priorities. The goal of this analysis was to examine processes of engaging stakeholders in early implementation of EBQI to tailor VHA's medical home for women. Four inter-professional regional stakeholder planning meetings were conducted; these meetings engaged stakeholders by providing regional data about gender disparities in Veterans' care experiences. Subsequent to each meeting, qualitative interviews were conducted with 87 key stakeholders (leaders and staff). Stakeholders were asked to describe QI efforts and the use of data to change aspects of care, including women's health care. Interview transcripts were summarized and coded using a hybrid deductive/inductive analytic approach. The presentation of regional-level data about gender disparities resulted in heightened awareness and stakeholder buy-in and decision-making related to women's health-focused QI. Interviews revealed that stakeholders were familiar with QI, with regional and facility leaders aware of inter-disciplinary committees and efforts to foster organizational change, including PCMH transformation. These efforts did not typically focus on women's health, though some informal efforts had been undertaken. Barriers to engaging in QI included lack of communication across clinical service lines, fluidity in staffing, and lack of protected time. Inter-professional, multilevel stakeholders need to be engaged in implementation early, with data and discussion that convey the importance and relevance of a new initiative. Stakeholder perspectives on institutional norms (e.g., gender norms) and readiness for population-specific QI are useful drivers of clinical initiatives designed to transform care for clinical subpopulations.

Main outcomes of an RCT to pilot test reporting and feedback to foster research integrity climates in the VA

BACKGROUND

Assessing the integrity of research climates and sharing such information with research leaders may support research best practices. We report here results of a pilot trial testing the effectiveness of a reporting and feedback intervention using the Survey of Organizational Research Climate (SOuRCe).

METHODS

We randomized 41 Veterans Health Administration (VA) facilities to a phone-based intervention designed to help research leaders understand their survey results (enhanced arm) or to an intervention in which results were simply distributed to research leaders (basic arm). Primary outcomes were (1) whether leaders took action, (2) whether actions taken were consistent with the feedback received, and (3) whether responses differed by receptivity to quality improvement input.

RESULTS

Research leaders from 25 of 42 (59%) VA facilities consented to participate in the study intervention and follow-up, of which 14 were at facilities randomized to the enhanced arm. We completed follow-up interviews with 21 of the 25 leaders (88%), 12 from enhanced arm facilities. While not statistically significant, the proportion of leaders reporting taking some action in response to the feedback was twice as high in the enhanced arm than in the basic arm (67% vs. 33%, p = .20). While also not statistically significant, a higher proportion of actions taken among facilities in the enhanced arm were responsive to the survey results than in the basic arm (42% vs. 22%, p = .64).

CONCLUSIONS

Enhanced feedback of survey results appears to be a promising intervention that may increase the likelihood of responsive action to improve organizational climates. Due to the small sample size of this pilot study, even large percentage-point differences between study arms are not statistically distinguishable. This hypothesis should be tested in a larger trial.