Methods proposed for monitoring the implementation of evidence-based research: a cross-sectional study

OBJECTIVES

Evidence-based research (EBR) is the systematic and transparent use of prior research to inform a new study so that it answers questions that matter in a valid, efficient, and accessible manner. This study surveyed experts about existing (e.g., citation analysis) and new methods for monitoring EBR and collected ideas about implementing these methods.

STUDY DESIGN AND SETTING

We conducted a cross-sectional study via an online survey between November 2022 and March 2023. Participants were experts from the fields of evidence synthesis and research methodology in health research. Open-ended questions were coded by recurring themes; descriptive statistics were used for quantitative questions.

RESULTS

Twenty-eight expert participants suggested that citation analysis should be supplemented with content evaluation (not just what is cited but also in which context), content expert involvement, and assessment of the quality of cited systematic reviews. They also suggested that citation analysis could be facilitated with automation tools. They emphasized that EBR monitoring should be conducted by ethics committees and funding bodies before the research starts. Challenges identified for EBR implementation monitoring were resource constraints and clarity on responsibility for EBR monitoring.

CONCLUSION

Ideas proposed in this study for monitoring the implementation of EBR can be used to refine methods and define responsibility but should be further explored in terms of feasibility and acceptability. Different methods may be needed to determine if the use of EBR is improving over time.

Conclusiveness, readability and textual characteristics of plain language summaries from medical and non-medical organizations: a cross-sectional study

This cross-sectional study compared plain language summaries (PLSs) from medical and non-medical organizations regarding conclusiveness, readability and textual characteristics. All Cochrane (medical PLSs, n = 8638) and Campbell Collaboration and International Initiative for Impact Evaluation (non-medical PLSs, n = 163) PLSs of latest versions of systematic reviews published until 10 November 2022 were analysed. PLSs were classified into three conclusiveness categories (conclusive, inconclusive and unclear) using a machine learning tool for medical PLSs and by two experts for non-medical PLSs. A higher proportion of non-medical PLSs were conclusive (17.79% vs 8.40%, P < 0.0001), they had higher readability (median number of years of education needed to read the text with ease 15.23 (interquartile range (IQR) 14.35 to 15.96) vs 15.51 (IQR 14.31 to 16.77), P = 0.010), used more words (median 603 (IQR 539.50 to 658.50) vs 345 (IQR 202 to 476), P < 0.001). Language analysis showed that medical PLSs scored higher for disgust and fear, and non-medical PLSs scored higher for positive emotions. The reason for the observed differences between medical and non-medical fields may be attributed to the differences in publication methodologies or disciplinary differences. This approach to analysing PLSs is crucial for enhancing the overall quality of PLSs and knowledge translation to the general public.

Author instructions in biomedical journals infrequently address systematic review reporting and methodology: a cross-sectional study

OBJECTIVES

We aimed to analyze how instructions for authors in journals indexed in MEDLINE address systematic review (SR) reporting and methodology.

STUDY DESIGN AND SETTING

We analyzed instructions for authors in 20% of MEDLINE-indexed journals listed in the online catalog of the National Library of Medicine on July 27, 2021. We extracted data only from the instructions published in English. We extracted data on the existence of instructions for reporting and methodology of SRs.

RESULTS

Instructions from 1,237 journals mentioned SRs in 45% (n = 560) of the cases. Systematic review (SR) registration was mentioned in 104/1,237 (8%) of instructions. Guidelines for reporting SR protocols were found in 155/1,237 (13%) of instructions. Guidelines for reporting SRs were explicitly mentioned in 461/1,237 (37%), whereas the EQUATOR (Enhancing the Quality and Transparency of Health Research) network was referred to in 474/1,237 (38%) of instructions. Less than 2% (n = 20) of instructions mentioned risk of bias and meta-analyses; less than 1% mentioned certainty of evidence assessment, methodological expectations, updating of SRs, overviews of SRs, or scoping reviews.

CONCLUSION

Journals indexed in MEDLINE rarely provide instructions for authors regarding SR reporting and methodology. Such instructions could potentially raise authors' awareness and improve how SRs are prepared and reported.

The definition of research misconduct should be stated in the abstract when reporting research on research misconduct

Research integrity is the cornerstone for a reliable and trustworthy science. Research misconduct is classically defined as fabrication, falsification, or plagiarism. To be considered as such, the action must have been committed with the intent to mislead or deceive. There are many other research misbehaviors such as duplication, fake-peer review or lack of disclosure of conflicts of interest, that are often included in the definition of research misconduct in codes, policies, and professional documents. The definition of research misconduct varies among countries and institutions, the seriousness and intentionality of the action. This variability is also present in research articles on the prevalence of research misconduct because it is common for each author to use a different definition, creating confusion for readers. We argue that the definition of research misconduct used in a study should be stated already in the abstract, particularly because not all publications are in open access, so that readers can fully understand what the study found concerning research misconduct without needing to have access to the full article.

Research misconduct as a challenge for academic institutions and scientific journals

Research misconduct refers to a set of unethical practices in research and publication and is the main reason for retraction of articles published in the academic literature. Research misconduct has negative consequences and has generated public scepticism about research, which has led to increasing distrust in science. In this context, a joint effort by the scientific community, academic institutions, scientific journals and research funders is needed to identify and prevent research misconduct. In this paper, we will evaluate what has already been done and what is needed to do to better address research misconduct. The focus of this paper will be on the actions taken by academic institutions, as the first line of defence, and scientific journals, as the gatekeepers of science. However, scientific journals and academic institutions are only a part of a much larger and multistakeholder effort needed to address the challenges scientific research is facing.

Factors influencing the promotion and implementation of research integrity in research performing and research funding organizations: A scoping review

Promoting and implementing research integrity is considered the joint responsibility and effort of multiple stakeholders in the research community. We conducted a scoping review and analyzed 236 research articles and gray literature publications from biomedical sciences, social sciences, natural sciences (including engineering), and humanities that dealt with the factors that may positively or negatively impact the promotion and implementation of research integrity. Critical appraisal of evidence was performed for studies describing interventions aimed at research integrity promotion in order to provide insight into the effectiveness of these interventions. The results of this scoping review provide a comprehensive taxonomy of factors with positive or negative impact and their relatedness to individual researchers, research performing and funding organizations, and the system of science. Moreover, the results show that efforts for fostering and promoting research integrity should be implemented at all three levels (researcher, institution, system) simultaneously to deliver greater adherence and implementation of research integrity practices. Although various educational interventions aiming at research integrity promotion exist, we were not able to conclude on the effectiveness of explored interventions due to the methodological quality issues in the studies.

Evaluating health claim assessment skills of parents with preschool children: A cross-sectional study using Informed Health Choices Claim Evaluation Tool

Background

Health literacy is a global problem and is particularly relevant when making health care decisions about small children. We analysed how parents of preschool children assess health claims and explored the predictors of their assessment skill.

Methods

We conducted a survey with questions from the Claim Evaluation Tools (CET) database, part of the Informed Health Choices (IHC) project, in ten paediatric primary care practices of the Split-Dalmatia County Health Center, Split, Croatia, from 1 February to 31 March 2023. Eligible participants were parents accompanying preschool-aged children for check-ups. We also collected data on parents' and children's demographic and health characteristics (including the presence of any chronic illness in the child), visits to paediatric emergency service, hospitalisations, vaccination status, the presence of chronic illness of parents or relatives, and whether parents had to made treatment decisions for themselves and/or their family member.

Results

Overall, 402 parents of preschool children (median age 35 years (interquartile range (IQR) = 31.0-38.3)) had a median IHC CET test score of 10.0 (IQR = 8.0-11.0) out of 12 questions. The multiple regression analysis showed that female gender, higher level of education, being employed, and having a history of a visit to paediatric emergency service were significant predictors of the test score, explaining 21.9% of the variance.

Conclusions

Parents of preschool children have a very good ability for critical assessment of health-related statements in a complex health care system and an environment of generally unsatisfactory health literacy. Further studies should explore how parents understand health claims in different geographical, socio-economic and cultural setting, and explore educational interventions to increase critical thinking abilities and informed decision-making, especially among fathers, unemployed parents and those with lower levels of education.

Assessing the quality and completeness of reporting in health systems guidance for pandemics using the AGREE-HS tool

Background

During health emergencies, leading healthcare organisations, such as the World Health Organization (WHO), the European Centre for Disease Control and Prevention (ECDC), and the United States Centers for Disease Control and Prevention (CDC), provide guidance for public health response. Previous studies have evaluated clinical practice guidelines (CPGs) produced in response to epidemics or pandemics, yet few have focused on public health guidelines and recommendations. To address this gap, we assessed health systems guidance (HSG) produced by the WHO, the ECDC, and the CDC for the 2009 H1N1 and COVID-19 pandemics.

Methods

We extracted HSG for the H1N1 and COVID-19 pandemics from the organisations' dedicated repositories and websites. After screening the retrieved documents for eligibility, five assessors evaluated them using the Appraisal of Guidelines Research & Evaluation - Health Systems (AGREE-HS) tool to assess the completeness and transparency of reporting according to the five AGREE-HS domains: "Topic", "Participants", "Methods", "Recommendations", and "Implementability".

Results

Following the screening process, we included 108 HSG in the analysis. We observed statistically significant differences between the H1N1 and COVID-19 pandemics, with HSG issued during COVID-19 receiving higher AGREE-HS scores. The HSG produced by the CDC had significantly lower overall scores and single-domain scores compared to the WHO and ECDC. However, all HSG scored relatively low, under the median of 40 total points (range = 10-70), indicating incomplete reporting. The HSG produced by all three organisations received a median score <4 (range = 1-7) for the "Participants", "Methods", and "Implementability" domains.

Conclusions

There is still significant progress to be made in the quality and completeness of reporting in HSG issued during pandemics, especially regarding methodological approaches and the composition of the guidance development team. Due to their significant impact and importance for healthcare systems globally, HSG issued during future healthcare crises should adhere to best reporting practices to increase uptake by stakeholders and ensure public trust in healthcare organisations.

Tool to assess recognition and understanding of elements in Summary of Findings Table for health evidence synthesis: a cross-sectional study

of Findings (SoF) tables concisely present the main findings of evidence synthesis of health evidence, but how users navigate it to understand and interpret the presented information is not clear. We quantified the interaction of medical students with an SoF table while answering a knowledge quiz. Read&Learn tool was used to measure the number of target and non-target table cells visited for each question and the time spent on these cells. Students positively identified target elements for quiz questions and answered simpler questions, but struggled with critical thinking and understanding study outcomes. The question on outcomes with the largest improvement post-intervention had the fewest correct answers, the longest interaction with table cells and the most opened cells before answering. Students spent a median of 72% of the time reading target table cells. A heatmap of the interactions showed that they were mostly answer-oriented. Further development of the tool and metrics is needed to use the tool and the metrics to study the cognitive processes during the assessment of health evidence.

Completeness of intervention description in invasive cardiology trials: an observational study of ClinicalTrials.gov registry and corresponding publications

Introduction

Non-pharmacological invasive interventions in cardiology are complex and often inadequately reported. Template for Intervention Description and Replication (TIDieR) checklist and guide were developed to aid reporting and assessment of non-pharmacological interventions. The aim of our study was to assess the completeness of describing invasive cardiology interventions in clinical trials at the level of trial registration and corresponding journal article publication.

Methodology

We searched for clinical trials in invasive cardiology registered in Clinicaltrials.gov and corresponding journal publications. We used the 10-item TIDieR checklist for registries and 12-item checklist for journal publications.

Results

Out of 7,017 registry items retrieved by our search, 301 items were included in the analysis. The search for corresponding published articles yielded 192 journal publications. The majority of trials were funded by the industry and were medical device trials. The median number of reported TIDieR items was 4.5 (95% CI 4.49-4.51) out of 10, and while the corresponding journal articles reported 6.5 (95% CI 6.0-6.5) out of 12 TIDieR items.

Conclusion

Registration and reporting of invasive cardiology trials is often incomplete and adequate detailed description of the interventions is not provided. TIDieR checklist is an important tool which should be used to ensure rigorous reporting of non-pharmacological interventions in cardiology.

Reporting characteristics of nonsurgical periodontal therapy trials registered in ClinicalTrials.gov: an observational study

Aim: To evaluate the completeness of the description of nonsurgical periodontal therapy interventions in clinical trials registered in ClinicalTrials.gov and correspondence of registered information for trial participants and outcome measures with published articles. Materials & methods: We retrieved data from ClinicalTrials.gov and corresponding publications. The completeness of intervention reporting was assessed using the Template for Intervention Description and Replication (TIDieR) checklist for oral hygiene instructions (OHI), professional mechanical plaque removal (PMPR), and subgingival instrumentation, antiseptics and antibiotics. The completeness of registration of trial protocol information was assessed according to the WHO Trial Registration DataSet for participant information (enrollment, sample size calculation, age, gender, condition) and primary/secondary outcome measures. Results: 79 included trials involved OHI (n = 38 trials, 48.1%), PMPR (n = 19, 24.1%), antiseptics (n = 11, 12.7%), or antibiotics (n = 11, 12.7%). There was a great variety in the terms used to describe these interventions. Most of the analyzed trials (93.7%) were completed and did not provide any data on study phase (74.7%). The description of intervention in the registry in ClinicalTrials.gov was inadequate for all analyzed interventions, with description inconsistencies in matching publications. There were also discrepancies in registered and published outcomes: for 39 trials with published results, 18 had different registered and reported primary outcomes, and 29 different registered and reported secondary outcomes. Conclusion: The completeness of the description of nonsurgical therapy of periodontitis in clinical trials is unsatisfactory, reducing the quality of translation of the new evidence and procedures into clinical practice. Significant discrepancy in registered and reported trial outcomes calls into question the validity of reported results and relevance for practice.

Linguistic analysis of plain language summaries and corresponding scientific summaries of Cochrane systematic reviews about oncology interventions

BACKGROUND

Cochrane plain language summaries (PLSs) are an important format to present high-quality healthcare evidence to patients with cancer and their families. They should be written in a way everyone can understand, since they serve as a tool in decision-making and present a bridge to overcome the gap between the healthcare users and professionals.

OBJECTIVE

The aim of the study was to assess the language characteristics of PLSs of Cochrane systematic reviews of oncology interventions in comparison with corresponding Cochrane scientific abstracts (SAs).

METHODS

In this cross-sectional study, we included all Cochrane PLSs and SAs of systematic reviews of oncology interventions available in the Cochrane Database of Systematic Reviews. We assessed text readability, measured using the Simple Measure of Gobbledygook (SMOG) index, and the prevalence of words related to different language tones (clout, authenticity, emotions and analytical tones). Two independent assessors categorized the conclusiveness of the efficacy of interventions into nine categories.

RESULTS

The overall median SMOG index for 275 PLSs was 13.0 (95% confidence interval [CI] 12.8-13.3). Readability scores did not differ across Cochrane Review Groups. SAs had a higher readability index than the corresponding PLSs (median = 16.6, 95% CI = 16.4-16.8). Regarding linguistic characteristics, PLSs were shorter than SAs, with less use of analytical tone, but more use of a positive emotional tone and authenticity. Overall, the 'Unclear' category of conclusiveness was the most common among all PLSs. Also, PLSs with 'No evidence' conclusions were the shortest and had the lowest SMOG index.

CONCLUSION

PLSs of Cochrane systematic reviews of oncological interventions have low readability and most give unclear conclusions about the efficacy of interventions. PLSs should be simplified so that patients and their families can benefit from appropriate health information on evidence synthesis. Further research is needed into reasons for unclear language to describe evidence from oncology trials.

Student Moral Foundations in the Context of Research Misbehavior: An Experimental Study

The study aimed to examine whether the use of words related to different moral foundations can predict the perceived severity of research misconduct. We gave two groups of participants, undergraduate medical students, the same hypothetical scenarios of research misconduct cases containing words related to different aspects of morality, and asked them to assess how inappropriate the described behavior was. Students ranked the described behaviors differently by the inappropriateness of the behavior, but the group wording was not a significant predictor of appropriateness. The reasons for the ranking were not related to any standardized procedures for research integrity but were related to the moral assessment of the students, which was assessed using qualitative approach. The results of this study implicate that personal moral views are an important part in research integrity training.

Delphi survey on the most promising areas and methods to improve systematic reviews' production and updating

BACKGROUND

Systematic reviews (SRs) are invaluable evidence syntheses, widely used in biomedicine and other scientific areas. Tremendous resources are being spent on the production and updating of SRs. There is a continuous need to automatize the process and use the workforce and resources to make it faster and more efficient.

METHODS

Information gathered by previous EVBRES research was used to construct a questionnaire for round 1 which was partly quantitative, partly qualitative. Fifty five experienced SR authors were invited to participate in a Delphi study (DS) designed to identify the most promising areas and methods to improve the efficient production and updating of SRs. Topic questions focused on which areas of SRs are most time/effort/resource intensive and should be prioritized in further research. Data were analysed using NVivo 12 plus, Microsoft Excel 2013 and SPSS. Thematic analysis findings were used on the topics on which agreement was not reached in round 1 in order to prepare the questionnaire for round 2.

RESULTS

Sixty percent (33/55) of the invited participants completed round 1; 44% (24/55) completed round 2. Participants reported average of 13.3 years of experience in conducting SRs (SD 6.8). More than two thirds of the respondents agreed/strongly agreed the following topics should be prioritized: extracting data, literature searching, screening abstracts, obtaining and screening full texts, updating SRs, finding previous SRs, translating non-English studies, synthesizing data, project management, writing the protocol, constructing the search strategy and critically appraising. Participants have not considered following areas as priority: snowballing, GRADE-ing, writing SR, deduplication, formulating SR question, performing meta-analysis.

CONCLUSIONS

Data extraction was prioritized by the majority of participants as an area that needs more research/methods development. Quality of available language translating tools has dramatically increased over the years (Google translate, DeepL). The promising new tool for snowballing emerged (Citation Chaser). Automation cannot substitute human judgement where complex decisions are needed (GRADE-ing).

TRIAL REGISTRATION

Study protocol was registered at https://osf.io/bp2hu/ .

Researchers on research integrity: a survey of European and American researchers

BACKGROUND

Reports of questionable or detrimental research practices (QRPs) call into question the reliability of scientific evidence and the trustworthiness of research. A critical component of the research ecosystem is the organization within which research takes place. We conducted a survey to explore the attitudes and beliefs of European and American researchers about the organisations in which they work, their own research practices and their attitudes towards research integrity and research integrity policies.

METHODS

We administered an online survey (International Research Integrity Survey (IRIS)) to 2,300 active researchers based in the US and 45,000 in Europe (including UK, Norway, Iceland and Switzerland).  We employed a stratified probability sample of the authors of research articles published between 2016 and 2020 included in Clarivate's Web of Science citation database. Coverage includes researchers in the humanities, social sciences, natural sciences and medical sciences, who hold at least a master's level degree.

RESULTS

In comparison to researchers in the US, European researchers admit to more QRPs and are less confident in maintaining high research integrity (RI) standards. In the US and Europe, many researchers judge their organization to fall short of best RI practice. All researchers recognize the benefits of RI, reliable knowledge and the trust of colleagues and the public, and there is support for RI training particularly among Europeans.

CONCLUSION

To create and maintain a culture of integrity in scientific research, a collective commitment from researchers, their institutions and funders is needed. Researchers rely on many channels of communication about research integrity and thus the involvement of many different participants in the research system is required to make improvements. Policies must be developed to reinforce best practice rather than being seen as an irrelevance to the real business of research.

Perception of organizational climate by university staff and students in medicine and humanities: A qualitative study

Organizational climate and culture are important for research organizations because they foster research integrity and responsible conduct of research, reduce questionable research practices, and improve job satisfaction. The aim of our study was to explore how employees and students perceive organizational climate and its consequences in the university setting. We conducted semi-structured interviews with senior students and employees (teaching and non-teaching staff) from two different university schools: School of Medicine and Faculty of Humanities and Social Sciences. Participants were asked questions regarding perceived climate, working environment, and the role of the institution. The data were analyzed using reflexive thematic analysis approach. Three themes were identified. The first theme addressed the difference in the perception and understanding of organizational climate. The second theme dealt with institutional issues emanating from organizational climate. The third theme described the behavior of stakeholders in the formation of organizational climate. Organizational climate is important concept in academic organizations as it influences both employees, particularly early career researchers, and students. Institutional leadership can strongly influence organizational climate, which can in turn affect job and job satisfaction. Due to the importance of personal morality on everyday decision-making, virtue-based research integrity training could be useful in improving academic institutions' organizational climate.

Creating research ethics and integrity country report cards: Case study from Europe

Structures for and practices of research integrity (RI) and research ethics (RE) differ among countries. This study analyzed the processes and structures for RI and RE in Europe, following the framework developed at the World Conferences on Research Integrity. We present RI and RE Country Report Cards for 16 European countries, which included the information on RI and RE structures, processes and outcomes. While some of the countries are front-runners when it comes to RI and RE, with well-established and continually developing policies and structures, others are just starting their journey in RI and RE. Although RI and RE contextual divergences must be taken into account, a level of harmonization among the countries is necessary so that researchers working in the European area can similarly handle RI and RE issues and have similar expectations regardless of the organization in which they work. RI and RE Country Report Cards can be a tool to monitor, compare, and strengthen RE and integrity across countries through empowerment and inspiration by examples of good practices and developed systems.

The Embassy of Good Science - a community driven initiative to promote ethics and integrity in research

The Embassy of Good Science ( https://www.embassy.science) aims to improve research integrity and research ethics by offering an online, open, 'go-to' platform, which brings together information on research integrity and research ethics and makes that information accessible, understandable, and appealing. It effectively organizes and describes research integrity and research ethics guidelines, educational materials, cases, and scenarios. The Embassy is wiki-based, allowing users to add -- when logged in with their ORCID researcher id -- new information, and update and refine existing information. The platform also makes the research integrity and research ethics community visible and more accessible in pages dedicated to relevant initiatives, news and events. Therefore, the Embassy enables researchers to find useful guidance, rules and tools to conduct research responsibly. The platform empowers researchers through increased knowledge and awareness, and through the support of the research integrity and research ethics community. In this article we will discuss the background of this new platform, the way in which it is organized, and how users can contribute.

Diplomatic response to global health challenges in recognizing patient needs: A qualitative interview study

Background

Global health diplomacy is the applied practice of foreign affairs to further national goals that focus on health issues requiring international cooperation and collective action. We aimed to determine how international diplomats and health policy-related professionals in the EU understand the concept of health diplomacy, which impacts both diplomatic relations as well as patients' rights.

Methods

In a qualitative interview study, we used a heterogeneous stratified purposeful sampling to reach participants from different countries and different practitioners from the Pyramid of Health Diplomacy: core, multi-stakeholder, and informal. Reflexive thematic analysis was used to identify the main themes.

Findings

We contacted 131 practitioners of GHD, of which 37 responded, and nine agreed to be interviewed. From 11 interview questions, four main themes emerged from the analysis of the individual interview. The participants reported limited knowledge about the definition of GHD but also that they engaged in daily activities and decisions of inter-governmental bodies. They were not aware of existing special education and training for health attachés and made suggestions for improving the field and practice of GHD. They were not fully familiar with the European Charter of Patients' Rights. There was a consensus from all participants that patient rights need to improve as a fundamental right. They stressed the fact that the hospital lockdown and the right access to healthcare were impaired during the COVID pandemic.

Interpretation

The role of health diplomacy in linking public health and foreign affairs is key to respecting patients' rights. Health over other interests is becoming an increasingly critical element in foreign policy. Establishing a clear career path for health attachés is necessary to foster effective global health agreements and coordination across countries.

Effectiveness of letters to patients with or without Cochrane blogshots on 10-year cardiovascular risk change among women in menopausal transition: 6-month three-arm randomized controlled trial

BACKGROUND

Health information and patient education on lifestyle changes may have a positive effect on the prevention of many chronic conditions, especially cardiovascular diseases (CVDs). We performed a parallel, three-arm randomized controlled trial (RCT) of 6-month educational intervention in a form of letters containing a reminder of the participant's CVD risk with or without Cochrane blogshots to reduce CVD risk among women aged 45-65 with one or more known CVD risk factors.

METHODS

The control group received a letter about their CVD risk at the beginning of the trial. The intervention groups received the initial letter about their CVD risk and remainder letters about their CVD risk every 2 months, with or without Cochrane blogshots: (1) effect of calcium in the prevention of high blood pressure, (2) effect of reducing saturated fat acids in eating habits, and (3) effects of green and black tea in CVD prevention. The primary outcome was CVD risk reduction calculated as the difference between the baseline and 6-month score for a 10-year risk of fatal CVD according to the ACC/AHA guidelines.

RESULTS

After both interventions, CVD risk reduction was significantly higher compared to the control group (P < 0.001, Kruskal-Wallis H test). The number of participants who decreased their CV risk was 29% (20/70) in the control group, 69% (48/70) in the group receiving the reminder letters, and 70% (49/70) in the group receiving the reminder letters and blogshots. The number needed to treat to achieve risk reduction was 2.41 (95% CI = 1.77 to 3.78) for letters with a CVD risk reminder and 2.50 (1.81 to 4.03) for letters with a reminder and a blogshot. The group receiving reminder letters with Cochrane blogshots had a significant change in the category of CVD risk, mainly from high to moderate and from moderate to low CVD risk category.

CONCLUSIONS

A simple and inexpensive intervention method in a form of letters reminding women about their CVD risk with or without providing additional health information in the form of Cochrane blogshots about interventions for important CVD risk factors may be effective in CVD management and could be considered by primary care providers.

TRIAL REGISTRATION

ClinicalTrials.gov, NCT04601558. Retrospectively registered on October 19, 2020.

Creating an online educational intervention to improve knowledge about systematic reviews among healthcare workers: mixed-methods pilot study

BACKGROUND

Lack of knowledge about systematic reviews (SRs) could prevent individual healthcare workers (HCWs) from using SRs as a source of information in their clinical practice or discourage them from participating in such research. In this study, we aimed to explore in-depth the opinion of a sample of HCWs about the newly created online educational intervention designed to improve knowledge about SRs.

METHODS

We created a brief online educational intervention on SRs, consisting of 11 textual modules. We evaluated it among practicing HCWs who graduated from a university-level health sciences program using a mixed-methods pilot study that consisted of pre- and post-intervention questionnaires and qualitative evaluation via semi-structured interviews. We assessed participants' knowledge about SR methodology before and after the intervention, and compared the responses. We sought their opinions about the characteristics of SRs. Also, the participants were presented with four scientific abstracts, where they were asked to distinguish whether those abstracts presented summaries of a systematic or a non-systematic review.

RESULTS

Twelve participants took part in the study. In the pilot study, the participants' knowledge about SRs was improved after the intervention compared to the baseline. Participants provided positive feedback regarding the educational intervention. Suggestions to improve the educational intervention were to provide more details about the forest plot, add more digital content or images, provide more details about the methodological steps of an SR, add descriptions about practical applications of SRs and provide links to additional educational materials. The participants suggested that HCWs could be motivated to take part in such an education if it is offered as continuing medical education (CME) course or credit for academic/career advancement.

CONCLUSION

HCWs provided positive feedback about the newly designed online educational intervention on SRs; they considered it an appropriate tool for learning about SRs and resulted in increased knowledge about SRs. In addition, participants gave suggestions for improving education, which can be used to tailor the education for end-users. In future studies, it would be useful to examine the effectiveness of the modified educational intervention on increasing knowledge in a larger sample and in the form of a randomized controlled trial.

Development of consensus on essential virtues for ethics and research integrity training using a modified Delphi approach

Most ethics and research integrity (ERI) training approaches are based on teaching moral rules, duties or responsibilities, often not sufficiently addressing virtue-based ethics. This study aimed to obtain a consensus among relevant experts on the importance of essential virtues for ERI training and their acquisitions. A modified Delphi consensus process was conducted in three rounds; 31 ERI experts participated in Round 1 and 23 in Round 2 and Round 3. Based on findings generated from qualitative data in Round 1, a structured questionnaire with 90 different statements grouped under five domains was developed for Round 2 and Round 3. After the final round, a consensus was achieved on two-thirds of statements included in this study. The experts agreed that virtues are based on learned and reflected attitudes and that the appropriate direction to acquire research virtues is through continuing education using case studies and discussions based on real-life scenarios. Furthermore, the consensus was obtained on 35 scientific virtues that should be stimulated in ERI training, prioritizing honesty, integrity, accountability, criticism and fairness as the most essential scientific virtues for good research practice. These results should be considered in developing or adjusting the ERI training program and materials.

Publishing Identifiable Patient Photographs in Scientific Journals: Scoping Review of Policies and Practices

Background

Publishing identifiable patient data in scientific journals may jeopardize patient privacy and confidentiality if best ethical practices are not followed. Current journal practices show considerable diversity in the publication of identifiable patient photographs, and different stakeholders may have different opinions of and practices in publishing patient photographs.

Objective

This scoping review aimed to identify existing evidence and map knowledge gaps in medical research on the policies and practices of publishing identifiable photographs in scientific articles.

Methods

We performed a comprehensive search of the Cochrane Central Register of Controlled Trials, Cochrane Database of Systematic Reviews, CINAHL with Full Text, Database of Abstracts of Reviews of Effects, Ovid MEDLINE, and Scopus. The Open Science Framework, PROSPERO, BASE, Google Scholar, OpenGrey, ClinicalTrials.gov, the Campbell Collaboration Library, and Science.gov were also searched.

Results

After screening the initial 15,949 titles and abstracts, 98 (0.61%) publications were assessed for eligibility at the full-text level, and 30 (0.19%) publications were included in this review. The studies were published between 1994 and 2020; most had a cross-sectional design and were published in journals covering different medical disciplines. We identified 3 main topics. The first included ethical aspects of the use of facial photographs in publications. In different clinical settings, the consent process was not conducted properly, and health professionals did not recognize the importance of obtaining written patient consent for taking and using patient medical photographs. They often considered verbal consent sufficient or even used the photographs without consent. The second topic included studies that investigated the practices and use of medical photography in publishing. Both patients and doctors asked for confidential storage and maintenance of medical photographs. Patients preferred to be photographed by their physicians using an institutional camera and preferred nonidentifiable medical photographs not only for publication but also in general. Conventional methods of deidentification of facial photographs concealing the eye area were recognized as unsuccessful in protecting patient privacy. The third topic emerged from studies investigating medical photography in journal articles. These studies showed great diversity in publishing practices regarding consent for publication of medical photographs. Journal policies regarding the consent process and consent forms were insufficient, and existing ethical professional guidelines were not fully implemented in actual practices. Patients’ photographs from open-access medical journals were found on public web-based platforms.

Conclusions

This scoping review showed a diversity of practices in publishing identifiable patient photographs and an unsatisfactory level of knowledge of this issue among different stakeholders despite existing standards. Emerging issues include the availability of patients’ photographs from open-access journals or preprints in the digital environment. There is a need to improve standards and processes to obtain proper consent to fully protect the privacy of patients in published articles.

Stakeholders' Experiences of Research Integrity Support in Universities: A Qualitative Study in Three European Countries

Fostering research integrity (RI) increasingly focuses on normative guidance and supportive measures within institutions. To be successful, the implementation of support should be informed by stakeholders' experiences of RI support. This study aims to explore experiences of RI support in Dutch, Spanish and Croatian universities. In total, 59 stakeholders (Netherlands n = 25, Spain n = 17, Croatia n = 17) participated in 16 focus groups in three European countries. Global themes on RI support experiences were identified by thematic analysis. Themes identified were: 'RI governance and institutional implementation', 'RI roles and structures', 'RI education and supervision', and 'Infrastructure, technology and tools supporting daily practice'. Experiences of support differed between countries in relation to: the efforts to translate norms into practice; the extent to which RI oversight was a responsibility of RE structures, or separate RI structures; and the availability of support close to research practice, such as training, responsible supervision, and adequate tools and infrastructure. The study reinforces the importance of a whole institutional approach to RI, embedded within local jurisdictions, rules, and practices. A whole institutional approach puts the emphasis of responsibility on institutions rather than individual researchers. When such an approach is lacking, some stakeholders look for intervention by authorities, such as funders, outside of the university.

Web-Based Educational Intervention to Improve Knowledge of Systematic Reviews Among Health Science Professionals: Randomized Controlled Trial

Background

Lack of knowledge of systematic reviews (SRs) could prevent individual health care professionals from using SRs as a source of information in their clinical practice or discourage them from participating in such research.

Objective

In this randomized controlled trial, we evaluated the effect of a short web-based educational intervention on short-term knowledge of SRs.

Methods

Eligible participants were 871 Master’s students of university health sciences studies in Croatia; 589 (67.6%) students who agreed to participate in the trial were randomized using a computer program into 2 groups. Intervention group A (294/589, 49.9%) received a short web-based educational intervention about SR methodology, and intervention group B (295/589, 50.1%) was presented with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) checklist. The participants’ knowledge of SRs was assessed before and after the intervention. The participants could not be blinded because of the nature of the intervention. The primary outcome was the difference in the percentage of correct answers about SR methodology per participant between the groups after the intervention, expressed as relative risk and 95% CI.

Results

Results from 162 and 165 participants in the educational intervention and PRISMA checklist groups, respectively, were available for analysis. Most of them (educational intervention group: 130/162, 80.2%; PRISMA checklist group: 131/165, 79.4%) were employed as health care professionals in addition to being health sciences students. After the intervention, the educational intervention group had 23% (relative risk percentage) more correct answers in the postintervention questionnaire than the PRISMA checklist group (relative risk=1.23, 95% CI 1.17-1.29).

Conclusions

A short web-based educational intervention about SRs is an effective tool for short-term improvement of knowledge of SRs among health care studies students, most of whom were also employed as health care professionals. Further studies are needed to explore the long-term effects of the tested education.

Trial Registration

OSF Registries 10.17605/OSF.IO/RYMVC; https://osf.io/rymvc

Designing and implementing a research integrity promotion plan: Recommendations for research funders

Various stakeholders in science have put research integrity high on their agenda. Among them, research funders are prominently placed to foster research integrity by requiring that the organizations and individual researchers they support make an explicit commitment to research integrity. Moreover, funders need to adopt appropriate research integrity practices themselves. To facilitate this, we recommend that funders develop and implement a Research Integrity Promotion Plan (RIPP). This Consensus View offers a range of examples of how funders are already promoting research integrity, distills 6 core topics that funders should cover in a RIPP, and provides guidelines on how to develop and implement a RIPP. We believe that the 6 core topics we put forward will guide funders towards strengthening research integrity policy in their organization and guide the researchers and research organizations they fund.

Inadequate reporting quality of registered genome editing trials: an observational study

Background

To assess registration completeness and safety data of trials on human genome editing (HGE) reported in primary registries and published in journals, as HGE has safety and ethical problems, including the risk of undesirable and unpredictable outcomes. Registration transparency has not been evaluated for clinical trials using these novel and revolutionary techniques in human participants.

Methods

Observational study of trials involving engineered site-specific nucleases and long-term follow-up observations, identified from the WHO ICTRP HGE Registry in November 2020 and two comprehensive reviews published in the same year. Registration and adverse events (AEs) information were collected from public registries and matching publications. Published data were extracted in May 2021.

Results

Among 81 eligible trials, most were recruiting (51.9%) phase 1 trials (45.7%). Five trials were withdrawn. Most trials investigated CAR T cells therapies (45.7%) and used CRISPR/Cas9 (35.8%) ex vivo (88.9%). Among 12 trials with protocols both registered and published, eligibility criteria, sample size, and secondary outcome measures were consistently reported for less than a half. Three trials posted results in ClinicalTrials.gov, and one reported serious AEs.

Conclusions

Incomplete registration and published data give emphasis to the need to increase the transparency of HGE trials. Further improvements in registration requirements, including phase 1 trials, and a more controlled publication procedure, are needed to augment the implementation of this promising technology.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-022-01574-0.

Research priorities to reduce the impact of COVID-19 in low- and middle-income countries

Background

The COVID-19 pandemic has caused disruptions to the functioning of societies and their health systems. Prior to the pandemic, health systems in low- and middle-income countries (LMIC) were particularly stretched and vulnerable. The International Society of Global Health (ISoGH) sought to systematically identify priorities for health research that would have the potential to reduce the impact of the COVID-19 pandemic in LMICs.

Methods

The Child Health and Nutrition Research Initiative (CHNRI) method was used to identify COVID-19-related research priorities. All ISoGH members were invited to participate. Seventy-nine experts in clinical, translational, and population research contributed 192 research questions for consideration. Fifty-two experts then scored those questions based on five pre-defined criteria that were selected for this exercise: 1) feasibility and answerability; 2) potential for burden reduction; 3) potential for a paradigm shift; 4) potential for translation and implementation; and 5) impact on equity.

Results

Among the top 10 research priorities, research questions related to vaccination were prominent: health care system access barriers to equitable uptake of COVID-19 vaccination (ranked 1st), determinants of vaccine hesitancy (4th), development and evaluation of effective interventions to decrease vaccine hesitancy (5th), and vaccination impacts on vulnerable population/s (6th). Health care delivery questions also ranked highly, including: effective strategies to manage COVID-19 globally and in LMICs (2nd) and integrating health care for COVID-19 with other essential health services in LMICs (3rd). Additionally, the assessment of COVID-19 patients’ needs in rural areas of LMICs was ranked 7th, and studying the leading socioeconomic determinants and consequences of the COVID-19 pandemic in LMICs using multi-faceted approaches was ranked 8th. The remaining questions in the top 10 were: clarifying paediatric case-fatality rates (CFR) in LMICs and identifying effective strategies for community engagement against COVID-19 in different LMIC contexts.

Interpretation

Health policy and systems research to inform COVID-19 vaccine uptake and equitable access to care are urgently needed, especially for rural, vulnerable, and/or marginalised populations. This research should occur in parallel with studies that will identify approaches to minimise vaccine hesitancy and effectively integrate care for COVID-19 with other essential health services in LMICs. ISoGH calls on the funders of health research in LMICs to consider the urgency and priority of this research during the COVID-19 pandemic and support studies that could make a positive difference for the populations of LMICs.

Predictors of academic progression and desire to continue education for undergraduate and graduate nursing students: Cross-sectional study and a nested follow-up study

BACKGROUND

There is conflicting evidence on predictors of nursing degree completion. Identification of predictors of student desire to continue their education could be important in developing strategies for encouraging and supporting students to complete their university-level nursing education.

AIM

To explore the factors associated with the enrollment in the first graduate year of nursing studies, progression from the first to the third year of university undergraduate nursing study, and the desire of third-year undergraduate students to continue nursing education at the graduate level.

PARTICIPANTS AND METHODS

A cross-sectional study involving 351 first- and third-year undergraduate and first-year graduate nursing students from three cohorts of the University of Split Department of Health Studies and one generation of first- and third-year undergraduate nursing students from the Catholic University of Croatia, Zagreb. We collected sociodemographic data on the participants, their opinions about their education, and scores on nine psychological questionnaires: performance self-efficacy, academic self-efficacy, perseverance, attitudes towards science, self-respect, motivation, dispositional hope, future time perspective, and perceived personal incompetence.

RESULTS

More than half of third-year undergraduate students wanted to continue their nursing studies at the graduate level, but they did not progress to the graduate degree studies immediately after finishing their undergraduate studies. None of the assessed characteristics predicted the desire of the final undergraduate year students to continue nursing education at the graduate level. In a nested follow-up study of a cohort of undergraduate students, we observed an increase in student attitudes towards science, extrinsic and intrinsic motivation, future orientation, and a decrease in perceived personal incompetence.

CONCLUSION

Our results demonstrate that immediate progression to graduate level of nursing education, although perceived as desired, is not achieved by undergraduate nursing students. Interventions focusing on motivation, future orientation, and personal competence need to be explored as a way to promote academic progression in nursing.

Ten years later: a review of the US 2009 institute of medicine report on conflicts of interest and solutions for further reform

Conflicts of interest (COIs) in healthcare are increasingly discussed in the literature, yet these relationships continue to influence healthcare. Research has consistently shown that financial COIs shape prescribing practices, medical education and guideline recommendations. In 2009, the Institute of Medicine (IOM, now the National Academy of Medicine) published Conflicts of Interest in Medical Research, Practice, and Education-one of the most comprehensive reviews of empirical research on COIs in medicine. Ten years after publication of theIOM's report, we review the current state of COIs within medicine. We also provide specific recommendations for enhancing scientific integrity in medical research, practice, education and editorial practices.

Deidentification of facial photographs: a survey of editorial policies and practices

We analysed all journals from two Journal Citation Reports (JCR) categories: 'Dentistry, Oral Surgery and Medicine' and 'Otorhinolaryngology' published in 2018 for their policies on publishing facial photographs and actual practices of publishing these photographs in articles. We extracted the following data for each journal: JCR category, impact factor, volume, issue, instructions for authors regarding ethical issues, instructions for photograph deidentification, journals' references to standard research and publishing policies, presence and type of published clinical images, separate informed consent for the publication of patient photograph and methods of deidentification. The sample included 103 journals, which published 568 articles with 1404 clinical images. Around a half of the journals (52%) had a policy on clinical images, however, the only predictor of having a journal policy on clinical images was reference in the policy to International Committee of Medical Journal Editors Recommendations (OR=3.00, 95% CI 1.26 to 7.14, p=0.013). Identifiable patient photographs were found in 13% (79/568) of the articles, constituting 9% (128/1404) of the total sample of images. Only 16% (13/79) of articles publishing recognisable patient facial images included a statement about consent for publication of the image. From the total sample of articles, 34% (27/79) contained deidentified but recognisable patient photographs and only 22% (6/27) of them had a statement about patient consent for photograph publication. The patients' consent was more likely stated in the article in cases of recognisable facial images (OR=2.81, 95% CI 1.41 to 5.63, p=0.004). Journals publishing clinical research involving the face and neck region need to establish and enforce policies on publishing clinical images.

Reporting violations of European Charter of Patients' Rights: analysis of patient complaints in Croatia

Background

The European Charter of Patients' Rights (ECPR) presents basic patients' rights in health care. We analysed the characteristics of patients' complaints about their rights submitted through the official complaints system and to a non-governmental organization in Croatia.

Methods

The official system for patients’complaints in Croatia does not have a common pathway but offers different modes for addressing patient complaints. In this cross-sectional study, we analysed the reports about patients’ complaints from the official regional committees sent to the Ministry of Health. We also analysed the complaints received by the Croatian Association for the Protection of Patient’s Rights (CAPR) and mapped them to the ECPR.

Results

The aggregated official data from the Ministry of Health in 2017 and 2018 covered only 289 individual complaints from 10 out of 21 counties. Complaints were most frequently related to secondary and tertiary healthcare institutions and details were not provided. CAPR received a total of 440 letters, out of which 207 contained 301 complaints about violations of patients’ rights in 2017–2018. The most common complaint was the Right of Access to health care (35.3%) from the ECPR, followed by the Right to Information (29.9%) and the Right to Safety (21.7%). The fewest complaints were about the Right to Complain (1.9%), Right to Innovation (1.4%), Right to Compensation (1.4%), and Right to Preventive Measures (1.0%).

Conclusions

Reporting and dealing with patients’ complaints about violations of their patients’ rights does not appear to be effective in a system with parallel but uncoordinated complaints pathways. Mapping patient's complaints to the ECPR is a useful tool to assess the perception of patients’ rights and to plan actions to improve the complaints system for effective health care.

Responsible research and innovation training programs: implementation and evaluation of the HEIRRI project

Responsible research and innovation, or RRI, is a concept that aims to bring together society and science for a better future. There are six key elements of RRI: public engagement, gender equality, science education, open access, ethics and governance.

Higher Education Institutions and Responsible Research and Innovation (HEIRRI) project aimed to bring the concept of RRI into the educational system. Using state-of-the-art review of good practices, HEIRRI team developed 10 training programs on RRI for different higher education institution educational levels, including a summer school and a massive open online course (MOOC). We conducted pilot of the trainings and evaluated participants’ experiences. Satisfaction with HEIRRI training programs on responsible research and innovation was high, both for participants and for the trainers, and trainings raised awareness of RRI. Participants’ feedback was used to identify areas that need improvement and provided for recommendations for final versions of the HEIRRI training programs.

In order to equip researchers with skills to recognize and apply RRI values, RRI should be included in their education. HEIRRI training is suitable for a range of different disciplines, including forensic science, and is free to use and adjust for specific contexts (available from: https://rri-tools.eu/heirri-training-programmes).

Supplemental data for this article is available online at https://doi.org/10.1080/20961790.2021.1970319 .

Development and implementation of research integrity guidance documents: Explorative interviews with research integrity experts

Research integrity (RI) guidance documents often lack sufficient details on handling specific RI issues causing the lack of harmonized approaches to RI and opening the way to research misconduct and other detrimental research practices. Standard operating procedures (SOPs) are developed and implemented by organizations for ensuring the uniformity and quality of performed actions. This study aimed to explore stakeholders' opinions on SOPs for RI, factors influencing the implementation of RI guidance documents and practices, and ideas for improvements in the RI field. We conducted semi-structured interviews with stakeholders from different groups. Data were analyzed using the reflexive thematic analysis approach, and three themes were developed. The first theme addressed participants' knowledge and perceptions on SOPs for RI and their impact on RI promotion and implementation. The second theme described different factors that have a positive or negative impact on the implementation of RI and RI guidance documents and practices, while the third theme addressed needed changes and ideas for improvements in the RI field. Participants considered SOPs valuable for RI promotion. SOPs should be developed based on and consistent with more general and aspirational guidance and through the dialogue with researchers and other stakeholders, to ensure their relevancy.

Antifragility of healthcare systems in Croatia and Bosnia and Herzegovina: Learning from man-made and natural crises

Acute crises, such as a war or a pandemic, are the ultimate tests for health care systems' resilience (temporary response to stress with change and adaptation) and antifragility (permanent benefit from change in response to stress). In this Health Policy paper, we analyse and discuss how the healthcare systems of two European countries - Bosnia and Herzegovina and Croatia - adapted to war as a man-made disaster, and how they adapted to COVID-19 pandemic twenty-five years later. These countries experienced full scale wars in recent history, which significantly changed their political and healthcare systems. This experience prepared the countries for the response to the pandemic, which coincided with two earthquakes in Croatia. We argue that healthcare systems in Croatia and Bosnia and Herzegovina are not only resilient but antifragile, and that they benefited from stressors they were exposed to. The antifragility of the two systems were primarily based on human effort - the strength, adaptability and resilience of health care professionals. We will look at lessons from the wars that were applied to the pandemic and discuss newly recognized opportunities and improvements.

Ethics issues identified by applicants and ethics experts in Horizon 2020 grant proposals

Background: We assessed the ethics review of proposals selected for funding under the Marie Skłodowska-Curie Actions (MSCA) and the European Research Council (ERC) in Horizon 2020, EU’s framework programme for research and innovation, 2014-2020.

Methods: We analysed anonymized datasets for 3,054 MSCA individual fellowships (IF), 417 MSCA Innovative Training Networks (ITN), and 1,465 ERC main-listed proposals with ethics conditional clearance, over four years (2016 to 2019). The datasets included the information on ethics issues identified by applicants in their proposal and ethics issues and requirements identified by ethics experts during the ethics review.

Results: 42% of proposals received ethical clearance. For proposals with conditional ethics clearance (n=3546), most of the identified ethics issues by both applicants and ethics experts were in the ethics categories related to humans; protection of personal data; environment, health and safety; and non-EU countries. Ethics experts identified twice as many ethics issues compared to applicants across funding schemes, years, and from high- and low-research performing countries. ERC grants had the highest number of ethics requirements per proposal (median (Md)=8, interquartile range (IQR=4-14), compared to ITN (Md=6, IQR=3-13) and IF grants (Md=3, IQR=2-6). The majority of requirements had to be fulfilled after grant agreement: 99.4% for IF, 99.5% for ITN, and 26.0% for ERC. For 9% of the proposals, the requirements included the appointment of an independent ethics advisor and 1% of the proposals had to appoint an ethics advisory board.

Conclusions: Many applicants for highly competitive H2020 funding schemes lack awareness of ethics issues raised by their proposed research. There is a need for better training of researchers at all career stages about ethics issues in research, more support to researchers from research organizations to follow the funding agencies requirements, as well as further development and harmonization of the ethics appraisal process during grant assessment.

Important Topics for Fostering Research Integrity by Research Performing and Research Funding Organizations: A Delphi Consensus Study

To foster research integrity (RI), it is necessary to address the institutional and system-of-science factors that influence researchers' behavior. Consequently, research performing and research funding organizations (RPOs and RFOs) could develop comprehensive RI policies outlining the concrete steps they will take to foster RI. So far, there is no consensus on which topics are important to address in RI policies. Therefore, we conducted a three round Delphi survey study to explore which RI topics to address in institutional RI policies by seeking consensus from research policy experts and institutional leaders. A total of 68 RPO and 52 RFO experts, representing different disciplines, countries and genders, completed one, two or all rounds of the study. There was consensus among the experts on the importance of 12 RI topics for RPOs and 11 for RFOs. The topics that ranked highest for RPOs concerned education and training, supervision and mentoring, dealing with RI breaches, and supporting a responsible research process (e.g. through quality assurance). The highest ranked RFO topics concerned dealing with breaches of RI, conflicts of interest, and setting expectations on RPOs (e.g. about educating researchers about RI). Together with the research policy experts and institutional leaders, we developed a comprehensive overview of topics important for inclusion in the RI policies of RPOs and RFOs. The topics reflect preference for a preventative approach to RI, coupled with procedures for dealing with RI breaches. RPOs and RFOs should address each of these topics in order to support researchers in conducting responsible research.

Does reviewing experience reduce disagreement in proposals evaluation? Insights from Marie Skłodowska-Curie and COST Actions

We have limited understanding of why reviewers tend to strongly disagree when scoring the same research proposal. Thus far, research that explored disagreement has focused on the characteristics of the proposal or the applicants, while ignoring the characteristics of the reviewers themselves. This article aims to address this gap by exploring which reviewer characteristics most affect disagreement among reviewers. We present hypotheses regarding the effect of a reviewer’s level of experience in evaluating research proposals for a specific granting scheme, that is, scheme reviewing experience. We test our hypotheses by studying two of the most important research funding programmes in the European Union from 2014 to 2018, namely, 52,488 proposals evaluated under three funding schemes of the Horizon 2020 Marie Sklodowska-Curie Actions (MSCA), and 1,939 proposals evaluated under the European Cooperation in Science and Technology Actions. We find that reviewing experience on previous calls of a specific scheme significantly reduces disagreement, while experience of evaluating proposals in other schemes—namely, general reviewing experience, does not have any effect. Moreover, in MSCA—Individual Fellowships, we observe an inverted U relationship between the number of proposals a reviewer evaluates in a given call and disagreement, with a remarkable decrease in disagreement above 13 evaluated proposals. Our results indicate that reviewing experience in a specific scheme improves reliability, curbing unwarranted disagreement by fine-tuning reviewers’ evaluation.

Perspectives of key stakeholders on essential virtues for good scientific practice in research areas

In contrast to the principle-based approach to ethics and research integrity (ERI) training, which stresses the importance of following moral rules, the virtue-based approach focuses on developing good character traits. This study has aimed to explore what virtues mean in scientific practice and their suitable place in ERI training, using a qualitative approach. Two face-to-face focus group discussions were conducted with 21 participants. Heterogeneous purposive sampling was used to reach participants from different countries, organization types (academia, research, publishing, private sector), scientific disciplines and stages of their scientific careers. Data generated during the focus group discussions were analyzed using a reflexive thematic analysis approach, and three main themes were developed. The first theme addressed the relativity of virtue meanings because the participants differed in their definitions and understandings of the concept of virtue. The second theme referred to the acquisition of virtues through social interactions because participants saw virtues mostly as social constructs acquired through socialization and education. The third theme addressed the differences in the importance of particular virtues in research. Participants felt that particular virtues were more important than others because some of them are necessary for responsible research, and some are not.

Efficacy of reminders for increasing volunteer engagement in translating Cochrane plain language summaries: a pilot randomised controlled trial

OBJECTIVES

The aim of this study was to pilot test the effectiveness of reminders versus no intervention for increasing the number of translated Cochrane plain language summaries (PLSs) among volunteer translators.

STUDY DESIGN

Parallel-group randomised controlled trial.

SETTING

Cochrane Croatia translation project.

PARTICIPANTS

Adults who volunteered to translate Cochrane PLSs within the Cochrane Croatia translation project.

INTERVENTION

The participants were randomly allocated to intervention (receiving up to four bi-weekly email reminders to translate PLSs) or control group (no intervention).

PRIMARY OUTCOME

The number of translated PLSs within the 6-month trial period.

RESULTS

We included 80 participants. The median number of translated PLSs after 6 months was 9 in the intervention group (95% CI 2.0 to 15.0) and 4 in the control group (95% CI 2.9 to 7.0), but this was not significantly different (p=0.181, Mann-Whitney U test). There was no difference between the groups in the number of translations after 3 months, the average time-to-translation after 3 or 6 months, or the satisfaction at the end of the study period. The number of reminders received and the number of translated summaries were negatively correlated (r=-0.50; 95% CI -0.70 to -0.22).

CONCLUSIONS

Our pilot trial showed that reminders do not seem to be significantly effective in increasing the number of PLS translations. Future studies could explore whether different frequency, timing and content of reminders have an influence on an increase in the engagement among volunteer translators of evidence synthesis.

TRIAL REGISTRATION

NCT03534791.