Assessing scientists for hiring, promotion, and tenure

A scoping review on what constitutes a good research culture

Background

The crisis in research culture is well documented, covering issues such as a tendency for quantity over quality, unhealthy competitive environments, and assessment based on publications, journal prestige and funding. In response, research institutions need to assess their own practices to promote and advocate for change in the current research ecosystem. The purpose of the scoping review was to explore ' What does the evidence say about the 'problem' with 'poor' research culture, what are the benefits of 'good' research culture, and what does 'good' look like?'

Aims

To examine the peer-reviewed and grey literature to explore the interplay between research culture, open research, career paths, recognition and rewards, and equality, diversity, and inclusion, as part of a larger programme of activity for a research institution.

Methods

A scoping review was undertaken. Six databases were searched along with grey literature. Eligible literature had relevance to academic research institutions, addressed research culture, and were published between January 2017 to May 2022. Evidence was mapped and themed to specific categories. The search strategy, screening and analysis took place between April-May 2022.

Results

1666 titles and abstracts, and 924 full text articles were assessed for eligibility. Of these, 253 articles met the eligibility criteria for inclusion. A purposive sampling of relevant websites was drawn from to complement the review, resulting in 102 records included in the review. Key areas for consideration were identified across the four themes of job security, wellbeing and equality of opportunity, teamwork and interdisciplinary, and research quality and accountability.

Conclusions

There are opportunities for research institutions to improve their own practice, however institutional solutions cannot act in isolation. Research institutions and research funders need to work together to build a more sustainable and inclusive research culture that is diverse in nature and supports individuals' well-being, career progression and performance.

Uncovering gaps in women's authorship: A big data analysis in academic surgery

INTRODUCTION

Women are underrepresented in surgical authorship. Using big data analyses, we aimed to investigate women's representation as first and last authors in surgical publications worldwide and identify underlying predictors.

METHODS

We retrieved eligible surgical journals using Scimago Journal & Country Rank 2021. We queried articles indexed in PubMed from selected journals published between January 2018 and April 2022. We used the EDirect tool to extract bibliometric data, including first and last authors' names, primary affiliation country, and publication year. Countries and dependent territories were classified following World Bank income levels and regions. Women's representation was predicted from forenames using the Gender-API software. Citations were included if gender accuracy was ≥80%.

RESULTS

We analyzed 210,853 citations containing both first and last authors' forenames, representing 158 countries and 14 territories. Women constituted 23.8% (50,161/210,853) of the first and 14.7% (31,069/210,853) of the last authors. High-income economies had more women as first authors than other income categories (p < 0.001), but fewer women as last authors than upper-middle- and lower-middle-income economies (p < 0.001). The odds of the first author being a woman were more than three times higher when the last author was also a woman (OR 3.21, 95% CI 3.13-3.30) and vice versa (OR 3.25, 95% CI 3.16-3.34) after adjusting for income level and publication year.

CONCLUSIONS

Women remain globally underrepresented in surgical authorship. Our findings urge concerted global efforts to overcome identified disparities.

Beyond Gender and Race: The Representation of Concealable Identities Among College Science Instructors at Research Institutions

Concealable stigmatized identities (CSIs) are identities that can be kept hidden and carry negative stereotypes. To understand the potential influence instructors have as role models, we must first explore the identities instructors have and whether they disclose those identities to undergraduates. We surveyed national samples of science instructors (n = 1248) and undergraduates (n = 2428) at research institutions to assess the extent to which instructors hold CSIs, whether they reveal those identities to undergraduates, how the prevalence of CSIs among instructors compares to their prevalence among undergraduates, and the reasons instructors reveal or conceal their CSIs. The most common CSIs instructors reported were having anxiety (35%) and being a first-generation college student (29%). Relatively few instructors revealed CSIs to students. The largest mismatches of CSI prevalence were for struggling academically in college (-30%) and having anxiety (-25%); all mismatches grew when accounting for instructor CSI disclosure, highlighting that students perceive fewer role models of scientists with CSIs than actually exist.

The SCOPE framework - implementing ideals of responsible research assessment

Background

Research and researchers are heavily evaluated, and over the past decade it has become widely acknowledged that the consequences of evaluating the research enterprise and particularly individual researchers are considerable. This has resulted in the publishing of several guidelines and principles to support moving towards more responsible research assessment (RRA). To ensure that research evaluation is meaningful, responsible, and effective the International Network of Research Management Societies (INORMS) Research Evaluation Group created the SCOPE framework enabling evaluators to deliver on existing principles of RRA. SCOPE bridges the gap between principles and their implementation by providing a structured five-stage framework by which evaluations can be designed and implemented, as well as evaluated.

Methods

SCOPE is a step-by-step process designed to help plan, design, and conduct research evaluations as well as check effectiveness of existing evaluations. In this article, four case studies are presented to show how SCOPE has been used in practice to provide value-based research evaluation.

Results

This article situates SCOPE within the international work towards more meaningful and robust research evaluation practices and shows through the four case studies how it can be used by different organisations to develop evaluations at different levels of granularity and in different settings.

Conclusions

The article demonstrates that the SCOPE framework is rooted firmly in the existing literature. In addition, it is argued that it does not simply translate existing principles of RRA into practice, but provides additional considerations not always addressed in existing RRA principles and practices thus playing a specific role in the delivery of RRA. Furthermore, the use cases show the value of SCOPE across a range of settings, including different institutional types, sizes, and missions.

A memory-theoretic account of citation propagation

Despite the common assumption that citations are indicative of an article's scientific merit, increasing evidence indicates that citation counts are largely driven by variables unrelated to quality. In this article, we treat people's decisions of what to cite as an instance of memory retrieval and show that observed citation patterns are well accounted for by a model of memory. The proposed exposure model anticipates that small alterations in factors that affect people's ability to retrieve to-be-cited articles from memory early in their life cycle are magnified over time and can lead to the emergence of highly cited papers. This effect occurs even when there is no variation in the starting point exposure probabilities (i.e. when assuming a level playing field where all articles are treated equally and of equal 'quality'), and is exacerbated by natural variation in retrievability of articles due to encoding. We discuss the implications of the model within the context of research evaluation and hiring, tenure and promotion decisions.

Twelve years after the ARRIVE guidelines: Animal research has not yet arrived at high standards

The reproducibility crisis across animal studies jeopardizes the credibility of the main findings derived from animal research, even though these findings are critical for informing human studies. To clarify and improve transparency among animal studies, the ARRIVE reporting guidelines were first announced in 2010 and upgraded to version 2.0 in 2020. However, compliance with and awareness of those reporting guidelines has remained suboptimal. Journal editors should encourage the authors to adhere to those guidelines. Authors, editors, referees, and reviewers should be aware of the ARRIVE guideline 2.0 when assessing and evaluating the methodology and findings of animal studies. However, we should also question whether reporting guidelines alone can change a research culture and improve the reproducibility of animal investigations. Reported research may not reflect actual research. Large segments of animal research efforts are wasted because of poor design choices and because of non-publication rather than suboptimal reporting. Better training of the scientific workforce, interventions at improving animal research at the design stage, registration practices, and alignment of the reward system with the publication of rigorous animal research may achieve more than reporting guidelines alone.

Automatically listing senior members of departments as co-authors is highly prevalent in health sciences: meta-analysis of survey research

A systematic review with meta-analysis was conducted to assess the prevalence of automatically listing (a) senior member(s) of a department as co-author(s) on all submitted articles in health sciences and the prevalence of degrees of support on a 5-point justification scale. Survey research was searched in PubMed, Lens.org, and Dimensions.ai. until January 5 2023. We assessed the methodological quality of studies and conducted quantitative syntheses. We identified 15 eligible surveys, that provided 67 results, all of which were rated as having low quality. A pooled estimate of 20% [95% CI 16-25] (10 surveys, 3619 respondents) of researchers in various health sciences reported that a senior member of their department was automatically listed as an author on all submitted articles. Furthermore, 28% [95% CI 22-34] of researchers (10 surveys, 2180 respondents) felt that this practice was 'never', 24% [95% CI 22-27] 'rarely', 25% [95% CI 23-28] 'sometimes', 13% [95% CI 9-17] 'most of the time', and 8% [95% CI 6-9] 'always justified'. The practice of automatically assigning senior members of departments as co-authors on all submitted manuscripts may be common in the health sciences; with those admitting to this practice finding it unjustified in most cases.Registration of the protocol The protocol was registered in Open Science Framework. Link: https://osf.io/4eywp/ .

Honorary authorship is highly prevalent in health sciences: systematic review and meta-analysis of surveys

A systematic review and meta-analysis of survey research was conducted to estimate honorary authorship prevalence in health sciences. We searched PubMed, Lens.org, and Dimensions.ai. until January 5 2023. Methodological quality was assessed and quantitative syntheses were conducted. Nineteen surveys were included and rated as having low methodological quality. We found a pooled prevalence of 26% [95% CI 21-31] (6 surveys, 2758 respondents) of researchers that perceived co-author(s) as honorary on the publication at issue (when they were not referred to any authorship criteria). That prevalence was 18% [95% CI 15-21] (11 surveys, 4272 respondents) when researchers were referred to Committee of Medical Journal Editors (ICMJE) authorship criteria, and 51% [95% CI 47-56] (15 surveys, 5111 respondents) when researchers were asked to declare their co-author(s) contributions on the publication at issue (and these were then compared to ICMJE criteria). 10% of researchers [95% CI 9-12] (11 surveys, 3,663 respondents) reported being approached by others to include honorary author(s) on the publication at issue and 16% [95% CI 13-18] (2 surveys, 823 respondents) admitted adding (an) honorary author(s). Survey research consistently indicates that honorary authorship in the health sciences is highly prevalent, however the quality of the surveys' methods and reporting needs improvement.

Mismatch in perceptions of the quality of supervision and research data management as an area of concern: Results from a university-wide survey of the research integrity culture at a Belgian university

Researchers of KU Leuven, a large Belgian university, were invited to complete a bespoke questionnaire assessing their attitudes toward research integrity and the local research culture, with specific emphasis on the supervision of junior researchers. A total of 7,353 invitations were sent via e-mail and 1,866 responses were collected (25.3% response rate), of which 1,723 responses are reported upon here. Some of the findings are relevant to the broader research community. Whereas supervisors evaluated their supervision of junior researchers almost unanimously as positive, fewer supervisees evaluated it as such. Data management emerged as an area of concern, both in terms of reviewing raw data and of data storage. More female than male professors emphasized open communication and supported their supervisees' professional development and personal well-being. At the same time, fewer female professors felt safe to speak up than male professors. Finally, researchers who obtained their master's degree outside Europe evaluated their supervision and KU Leuven's research culture more positively than researchers with a master's degree from KU Leuven. The results of the survey were fed back to the university's board and several bodies and served as input to update the university's research policy. Faculties and departments received a detailed report.

A Decade of Efforts to Add Value to Child Health Research Practices

OBJECTIVE

To identify practices that add value to improve the design, conduct, and reporting of child health research and reduce research waste.

STUDY DESIGN

In order to categorize the contributions of members of Standards for Research (StaR) in Child Health network, we developed a novel Child Health Improving Research Practices (CHIRP) framework comprised of 5 domains meant to counteract avoidable child health research waste and improve quality: 1) address research questions relevant to children, their families, clinicians, and researchers; 2) apply appropriate research design, conduct and analysis; 3) ensure efficient research oversight and regulation; 4) Provide accessible research protocols and reports; and 5) develop unbiased and usable research reports, including 17 responsible research practice recommendations. All child health research relevant publications by the 48 original StaR standards' authors over the last decade were identified, and main topic areas were categorized using this framework.

RESULTS

A total of 247 publications were included in the final sample: 100 publications (41%) in domain 1 (3 recommendations), 77 publications (31%) in domain 2 (3), 35 publications (14%) in domain 3 (4), 20 publications (8%) in domain 4 (4), and 15 publications (6%) in domain 5 (3). We identified readily implementable "responsible" research practices to counter child health research waste and improve quality, especially in the areas of patients and families' engagement throughout the research process, developing Core Outcome Sets, and addressing ethics and regulatory oversight issues.

CONCLUSION

While most of the practices are readily implementable, increased awareness of methodological issues and wider guideline uptake is needed to improve child health research. The CHIRP Framework can be used to guide responsible research practices that add value to child health research.

Early-career factors largely determine the future impact of prominent researchers: evidence across eight scientific fields

Can we help predict the future impact of researchers using early-career factors? We analyze early-career factors of the world's 100 most prominent researchers across 8 scientific fields and identify four key drivers in researchers' initial career: working at a top 25 ranked university, publishing a paper in a top 5 ranked journal, publishing most papers in top quartile (high-impact) journals and co-authoring with other prominent researchers in their field. We find that over 95% of prominent researchers across multiple fields had at least one of these four features in the first 5 years of their career. We find that the most prominent scientists who had an early career advantage in terms of citations and h-index are more likely to have had all four features, and that this advantage persists throughout their career after 10, 15 and 20 years. Our findings show that these few early-career factors help predict researchers' impact later in their careers. Our research thus points to the need to enhance fairness and career mobility among scientists who have not had a jump start early on.

Career development in pragmatic clinical trials to improve care for people living with dementia

The growing number of people living with dementia (PLWD) requires a coordinated clinical response to deliver pragmatic, evidence-based interventions in frontline care settings. However, infrastructure to support such a response is lacking. Moreover, there are too few researchers conducting rigorous embedded pragmatic clinical trials (ePCTs) to make the vision of high quality, widely accessible dementia care a reality. National Institute on Aging (NIA) Imbedded Pragmatic Alzheimer's disease and Related Dementias Clinical Trials (IMPACT) Collaboratory seeks to improve the pipeline of early career researchers qualified to lead ePCTs by funding career development awards. Even with support from the Collaboratory, awardees face practical and methodological challenges to success, recently exacerbated by the COVID-19 pandemic. We first describe the training opportunities and support network for the IMPACT CDA recipients. This report then describes the unique career development challenges faced by early-career researchers involved in ePCTs for dementia care. Topics addressed include challenges in establishing a laboratory, academic promotion, mentoring and professional development, and work-life balance. Concrete suggestions to address these challenges are offered for early-career investigators, their mentors, and their supporting institutions. While some of these challenges are faced by researchers in other fields, this report seeks to provide a roadmap for expanding the work of the IMPACT Collaboratory and initiating future efforts to recruit, train, and retain talented early-career researchers involved in ePCTs for dementia care.

The Association of Research Quantitative Measures With Faculty Ranks of Australian and New Zealand Dental Schools

INTRODUCTION

The scholarly productivity of a faculty member can be measured through several indicators including annual appraisals, feedback, and the number of publications per year. The present study aims to assess the association of quantitative research measures and academic ranks in Australian and New Zealand dental schools.

METHODS

It was an analytical observational cross-sectional study. Full-time faculty members working in Australia and New Zealand's dental schools were discovered on official websites. Various bibliometric parameters including h-index, total number of citations, total number of publications, and maximum number of sources of a single publication were analyzed. Spearman rank correlation was used to determine the correlation between bibliometric variables and academic ranks (lecturer, assistant professor, professor). The Mann-Whitney U test was used to compare bibliometric parameters among departments (Basic and Clinical) and gender (male and female).

RESULTS

Through the present search strategy, 207 full-time faculty members were identified, of which 12 were from New Zealand, and 195 were from Australia. Among them, 130 (62.8%) were male and 70 (33.8%) were female faculty members. There was a positive correlation of all bibliometric parameters with academic ranks (p = 0.001). There was no statistical difference between the two countries for academic parameters (p > 0.05). Male faculty members showed significantly higher academic productivity than female members in Australian dental schools (p = 0.001).

CONCLUSION

These bibliometric parameters and other educational parameters can be considered for determining faculty promotions. These bibliometric parameters appear to be suitable metrics for assessing research productivity, impact, and visibility.

Self-referencing rates in biological disciplines

The use of citation counts (among other bibliometrics) as a facet of academic research evaluation can influence citation behavior in scientific publications. One possible unintended consequence of this bibliometric is excessive self-referencing, where an author favors referencing their own publications over related publications from different research groups. Peer reviewers are often prompted by journals to determine whether references listed in the manuscript under review are unbiased, but there is no consensus on what is considered "excessive" self-referencing. Here, self-referencing rates are examined across multiple journals in the fields of biology, genetics, computational biology, medicine, pathology, and cell biology. Median self-referencing rates are between 8-13% across a range of journals within these disciplines. However, self-referencing rates vary as a function of total number of references, number of authors, author status/rank, author position, and total number of publications for each author. Importantly, these relationships exhibit interdisciplinary and journal-dependent differences that are not captured by examining broader fields in aggregate (e.g., Biology, Chemistry, Physics, etc.). These results provide useful statistical guidelines for authors, editors, reviewers, and journals when considering referencing practices for individual publications, and highlight the effects of additional factors influencing self-referencing rates.

Authorship and citation patterns of highly cited biomedical researchers: a cross-sectional study

BACKGROUND

Scientific productivity is often evaluated by means of cumulative citation metrics. Different metrics produce different incentives. The H-index assigns full credit from a citation to each coauthor, and thus may encourage multiple collaborations in mid-list author roles. In contrast, the Hm-index assigns only a fraction 1/k of citation credit to each of k coauthors of an article, and thus may encourage research done by smaller teams, and in first or last author roles. Whether H and Hm indices are influenced by different authorship patterns has not been examined.

METHODS

Using a publicly available Scopus database, I examined associations between the numbers of research articles published as single, first, mid-list, or last author between 1990 and 2019, and the H-index and the Hm-index, among 18,231 leading researchers in the health sciences.

RESULTS

Adjusting for career duration and other article types, the H-index was negatively associated with the number of single author articles (partial Pearson r -0.06) and first author articles (-0.08), but positively associated with the number of mid-list (0.64) and last author articles (0.21). In contrast, all associations were positive for the Hm-index (0.04 for single author articles, 0.18 for first author articles, 0.24 for mid-list articles, and 0.46 for last author articles).

CONCLUSION

The H-index and the Hm-index do not reflect the same authorship patterns: the full-credit H-index is predominantly associated with mid-list authorship, whereas the partial-credit Hm-index is driven by more balanced publication patterns, and is most strongly associated with last-author articles. Since performance metrics may act as incentives, the selection of a citation metric should receive careful consideration.

Barriers to Meaningful Participatory Mental Health Services Research and Priority Next Steps: Findings From a National Survey

OBJECTIVE

A growing consensus has emerged regarding the importance of stakeholder involvement in mental health services research. To identify barriers to and the extent of stakeholder involvement in participatory research, the authors undertook a mixed-methods study of researchers and community members who reported participation in such research.

METHODS

Eight consultative focus groups were conducted with diverse groups of stakeholders in mental health services research (N=51 unique participants, mostly service users), followed by a survey of service users, family members, community providers, and researchers (N=98) with participatory research experience. Focus groups helped identify facilitators and barriers to meaningful research collaboration, which were operationalized in the national survey. Participants were also asked about high-priority next steps.

RESULTS

The barrier most strongly endorsed as a large or very large problem in the field was lack of funding for stakeholder-led mental health services research (76%), followed by lack of researcher training in participatory methods (74%) and insufficiently diverse backgrounds among stakeholders (69%). The two most frequently identified high-priority next steps were ensuring training and continuing education for researchers and stakeholders (33%) and authentically centering lived experience and reducing tokenism in research (26%).

CONCLUSIONS

These findings suggest a need for increased attention to and investment in the development, implementation, and sustainment of participatory methods that prioritize collaboration with direct stakeholders, particularly service users, in U.S. mental health services research. The findings also underscore the presence and potentially important role of researchers who dually identify as service users and actively contribute a broader orientation from the service user-survivor movement.

Is Open Science Neoliberal?

The scientific-reform movement, frequently referred to as open science, has the potential to substantially reshape the nature of the scientific activity. For this reason, its sociopolitical antecedents and consequences deserve serious scholarly attention. In a recently formed literature that professes to meet this need, it has been widely argued that the movement is neoliberal. However, for two reasons it is hard to justify this widescale attribution: First, the critics mistakenly represent the movement as a monolithic structure, and second, the critics' arguments associating the movement with neoliberalism because of the movement's (a) preferential focus on methodological issues, (b) underlying philosophy of science, and (c) allegedly promarket ideological proclivities reflected in the methodology and science-policy proposals do not hold under closer scrutiny. These shortcomings show a lack of sufficient engagement with the reform literature. What is needed is more nuanced accounts of the sociopolitical underpinnings of scientific reform. To address this need, we propose a model for the analysis of reform proposals, which represents scientific methodology, axiology, science policy, and ideology as interconnected but relatively distinct domains, and thus allows for recognizing the divergent tendencies in the movement and the uniqueness of particular proposals.

Tackling the lack of diversity in cancer research

Despite the clear benefit of studying biological samples from diverse genetic backgrounds and geographical locations, our current knowledge of disease is mostly derived from the study of European-descent individuals. In the cancer field, this is reflected in the poor representation of African and Amerindian/Latino samples in most large public data repositories. This lack of diversity is due to several reasons, but here we focus on (1) the lack of support for studies on non-European populations that are performed in low- and middle-income countries (LMICs), and (2) unequal partnerships between scientists in LMICs and those in high-income countries. We argue that expanding access to research funding, increasing the participation of underrepresented scientists in editorial boards and international conferences, facilitating the publication of studies conducted in these countries, and properly acknowledging LMIC researchers' contributions in publications and grant applications will promote equity for scientists working in LMICs. We envisage that this will translate to more impactful research in these countries, which will include more samples from diverse populations. For the cancer field, this will broaden our understanding of pathomechanisms and may help to improve the treatment of patients from all backgrounds.

Ranking mobility and impact inequality in early academic careers

How difficult is it for an early career academic to climb the ranks of their discipline? We tackle this question with a comprehensive bibliometric analysis of 57 disciplines, examining the publications of more than 5 million authors whose careers started between 1986 and 2008. We calibrate a simple random walk model over historical data of ranking mobility, which we use to 1) identify which strata of academic impact rankings are the most/least mobile and 2) study the temporal evolution of mobility. By focusing our analysis on cohorts of authors starting their careers in the same year, we find that ranking mobility is remarkably low for the top- and bottom-ranked authors and that this excess of stability persists throughout the entire period of our analysis. We further observe that mobility of impact rankings has increased over time, and that such rise has been accompanied by a decline of impact inequality, which is consistent with the negative correlation that we observe between such two quantities. These findings provide clarity on the opportunities of new scholars entering the academic community, with implications for academic policymaking.

Impact of major awards on the subsequent work of their recipients

To characterize the impact of major research awards on recipients' subsequent work, we studied Nobel Prize winners in Chemistry, Physiology or Medicine, and Physics and MacArthur Fellows working in scientific fields. Using a case-crossover design, we compared scientists' citations, publications and citations-per-publication from work published in a 3-year pre-award period to their work published in a 3-year post-award period. Nobel Laureates and MacArthur Fellows received fewer citations for post- than for pre-award work. This was driven mostly by Nobel Laureates. Median decrease was 80.5 citations among Nobel Laureates (p = 0.004) and 2 among MacArthur Fellows (p = 0.857). Mid-career (42-57 years) and senior (greater than 57 years) researchers tended to earn fewer citations for post-award work. Early career researchers (less than 42 years, typically MacArthur Fellows) tended to earn more, but the difference was non-significant. MacArthur Fellows (p = 0.001) but not Nobel Laureates (p = 0.180) had significantly more post-award publications. Both populations had significantly fewer post-award citations per paper (p = 0.043 for Nobel Laureates, 0.005 for MacArthur Fellows, and 0.0004 for combined population). If major research awards indeed fail to increase (and even decrease) recipients' impact, one may need to reassess the purposes, criteria, and impacts of awards to improve the scientific enterprise.

The Concept of Scaffold-Guided Bone Regeneration for the Treatment of Long Bone Defects: Current Clinical Application and Future Perspective

The treatment of bone defects remains a challenging clinical problem with high reintervention rates, morbidity, and resulting significant healthcare costs. Surgical techniques are constantly evolving, but outcomes can be influenced by several parameters, including the patient's age, comorbidities, systemic disorders, the anatomical location of the defect, and the surgeon's preference and experience. The most used therapeutic modalities for the regeneration of long bone defects include distraction osteogenesis (bone transport), free vascularized fibular grafts, the Masquelet technique, allograft, and (arthroplasty with) mega-prostheses. Over the past 25 years, three-dimensional (3D) printing, a breakthrough layer-by-layer manufacturing technology that produces final parts directly from 3D model data, has taken off and transformed the treatment of bone defects by enabling personalized therapies with highly porous 3D-printed implants tailored to the patient. Therefore, to reduce the morbidities and complications associated with current treatment regimens, efforts have been made in translational research toward 3D-printed scaffolds to facilitate bone regeneration. Three-dimensional printed scaffolds should not only provide osteoconductive surfaces for cell attachment and subsequent bone formation but also provide physical support and containment of bone graft material during the regeneration process, enhancing bone ingrowth, while simultaneously, orthopaedic implants supply mechanical strength with rigid, stable external and/or internal fixation. In this perspective review, we focus on elaborating on the history of bone defect treatment methods and assessing current treatment approaches as well as recent developments, including existing evidence on the advantages and disadvantages of 3D-printed scaffolds for bone defect regeneration. Furthermore, it is evident that the regulatory framework and organization and financing of evidence-based clinical trials remains very complex, and new challenges for non-biodegradable and biodegradable 3D-printed scaffolds for bone regeneration are emerging that have not yet been sufficiently addressed, such as guideline development for specific surgical indications, clinically feasible design concepts for needed multicentre international preclinical and clinical trials, the current medico-legal status, and reimbursement. These challenges underscore the need for intensive exchange and open and honest debate among leaders in the field. This goal can be addressed in a well-planned and focused stakeholder workshop on the topic of patient-specific 3D-printed scaffolds for long bone defect regeneration, as proposed in this perspective review.

Responsible research practices could be more strongly endorsed by Australian university codes of research conduct

BACKGROUND

This study aimed to investigate how strongly Australian university codes of research conduct endorse responsible research practices.

METHODS

Codes of research conduct from 25 Australian universities active in health and medical research were obtained from public websites, and audited against 19 questions to assess how strongly they (1) defined research integrity, research quality, and research misconduct, (2) required research to be approved by an appropriate ethics committee, (3) endorsed 9 responsible research practices, and (4) discouraged 5 questionable research practices.

RESULTS

Overall, a median of 10 (IQR 9 to 12) of 19 practices covered in the questions were mentioned, weakly endorsed, or strongly endorsed. Five to 8 of 9 responsible research practices were mentioned, weakly, or strongly endorsed, and 3 questionable research practices were discouraged. Results are stratified by Group of Eight (n = 8) and other (n = 17) universities. Specifically, (1) 6 (75%) Group of Eight and 11 (65%) other codes of research conduct defined research integrity, 4 (50%) and 8 (47%) defined research quality, and 7 (88%) and 16 (94%) defined research misconduct. (2) All codes required ethics approval for human and animal research. (3) All codes required conflicts of interest to be declared, but there was variability in how strongly other research practices were endorsed. The most commonly endorsed practices were ensuring researcher training in research integrity [8 (100%) and 16 (94%)] and making study data publicly available [6 (75%) and 12 (71%)]. The least commonly endorsed practices were making analysis code publicly available [0 (0%) and 0 (0%)] and registering analysis protocols [0 (0%) and 1 (6%)]. (4) Most codes discouraged fabricating data [5 (63%) and 15 (88%)], selectively deleting or modifying data [5 (63%) and 15 (88%)], and selective reporting of results [3 (38%) and 15 (88%)]. No codes discouraged p-hacking or hypothesising after results are known.

CONCLUSIONS

Responsible research practices could be more strongly endorsed by Australian university codes of research conduct. Our findings may not be generalisable to smaller universities, or those not active in health and medical research.

Exploring the merits of research performance measures that comply with the San Francisco Declaration on Research Assessment and strategies to overcome barriers of adoption: qualitative interviews with administrators and researchers

BACKGROUND

In prior research, we identified and prioritized ten measures to assess research performance that comply with the San Francisco Declaration on Research Assessment, a principle adopted worldwide that discourages metrics-based assessment. Given the shift away from assessment based on Journal Impact Factor, we explored potential barriers to implementing and adopting the prioritized measures.

METHODS

We identified administrators and researchers across six research institutes, conducted telephone interviews with consenting participants, and used qualitative description and inductive content analysis to derive themes.

RESULTS

We interviewed 18 participants: 6 administrators (research institute business managers and directors) and 12 researchers (7 on appointment committees) who varied by career stage (2 early, 5 mid, 5 late). Participants appreciated that the measures were similar to those currently in use, comprehensive, relevant across disciplines, and generated using a rigorous process. They also said the reporting template was easy to understand and use. In contrast, a few administrators thought the measures were not relevant across disciplines. A few participants said it would be time-consuming and difficult to prepare narratives when reporting the measures, and several thought that it would be difficult to objectively evaluate researchers from a different discipline without considerable effort to read their work. Strategies viewed as necessary to overcome barriers and support implementation of the measures included high-level endorsement of the measures, an official launch accompanied by a multi-pronged communication strategy, training for both researchers and evaluators, administrative support or automated reporting for researchers, guidance for evaluators, and sharing of approaches across research institutes.

CONCLUSIONS

While participants identified many strengths of the measures, they also identified a few limitations and offered corresponding strategies to address the barriers that we will apply at our organization. Ongoing work is needed to develop a framework to help evaluators translate the measures into an overall assessment. Given little prior research that identified research assessment measures and strategies to support adoption of those measures, this research may be of interest to other organizations that assess the quality and impact of research.

Biomedical supervisors' role modeling of open science practices

Supervision is one important way to socialize Ph.D. candidates into open and responsible research. We hypothesized that one should be more likely to identify open science practices (here publishing open access and sharing data) in empirical publications that were part of a Ph.D. thesis when the Ph.D. candidates' supervisors engaged in these practices compared to those whose supervisors did not or less often did. Departing from thesis repositories at four Dutch University Medical centers, we included 211 pairs of supervisors and Ph.D. candidates, resulting in a sample of 2062 publications. We determined open access status using UnpaywallR and Open Data using Oddpub, where we also manually screened publications with potential open data statements. Eighty-three percent of our sample was published openly, and 9% had open data statements. Having a supervisor who published open access more often than the national average was associated with an odds of 1.99 to publish open access. However, this effect became nonsignificant when correcting for institutions. Having a supervisor who shared data was associated with 2.22 (CI:1.19-4.12) times the odds to share data compared to having a supervisor that did not. This odds ratio increased to 4.6 (CI:1.86-11.35) after removing false positives. The prevalence of open data in our sample was comparable to international studies; open access rates were higher. Whilst Ph.D. candidates spearhead initiatives to promote open science, this study adds value by investigating the role of supervisors in promoting open science.

One size does not fit all: an exploratory interview study on how translational researchers navigate the current academic reward system

Introduction

Translational research is a subfield of the biomedical life sciences that focuses on clinically driven healthcare innovations. The workforce of this subfield, i.e., translational researchers, are diversely specialized and collaborate with a multitude of stakeholders from diverse disciplines in and outside academia in order to navigate the complex path of translating unmet clinical needs into research questions and ultimately into advancements for patient care. Translational researchers have varying responsibilities in the clinical, educational, and research domains requiring them to split their time two- or three-ways. Working between these domains and alongside peers who do not split their time as such, raises questions about the academic reward system used to recognize their performance, which mainly focuses on publication metrics within the research domain. What is unclear is how combining research tasks with tasks in the clinical and/or educational domains effects translational researchers and how they navigate the academic reward system.

Methods

In this exploratory interview study, semi-structured interviews were conducted to gain a deeper understanding of the current academic reward system for translational researchers. Stratified purposeful sampling was used to recruit 14 translational researchers from varying countries, subspecialties, and career stages. The interviews were coded after data collection was complete and arranged into three overarching result categories: intrinsic motivation, extrinsic factors, and ideal academic reward system and advice.

Results

We found that these 14 translational researchers were intrinsically motivated to achieve their translational goals while working in settings where clinical work was reported to take priority over teaching which in turn took priority over time for research. However, it is the latter that was explained to be essential in the academic reward system which currently measures scientific impact largely based on publications metrics.

Conclusion

In this study, translational researchers were asked about their thoughts regarding the current academic reward system. Participants shared possible structural improvements and ideas for specialized support on an individual, institutional, and also international level. Their recommendations focused on acknowledging all aspects of their work and led to the conclusion that traditional quantitative academic reward metrics do not fully align with their translational goals.

Defining Scholarship for Today and Tomorrow

ABSTRACT

Scholarship, required for academic advancement, has traditionally been defined narrowly, not keeping pace with the expansion of faculty academic activities in health professions schools. How can we refine the definition of scholarship so that it better aligns with the scope of current faculty practice within academic health systems? Revision of the academic policies for promotion and tenure at the University of Massachusetts Chan Medical School afforded an opportunity to redefine scholarship such that a broader platform was available for faculty recognition, aligning with current academic standards, yet providing flexibility for the future. The authors describe the historical context of the definition of scholarship and their institution's process to construct a definition of scholarship with three essential elements: advancement of knowledge, dissemination for critical review, and impact on a discipline, practice, or community. Application of this definition to team science and digital scholarship is also described. Following a widespread continuing education initiative, implementation of the new definition within promotion and tenure processes of the medical, nursing, and graduate schools resulted in broad acceptance across the institution. This forum article provides lessons in leading an academic health sciences institution to reassess academic processes and is a resource for advancing the vigorous debate on the evolving meaning and evaluation of scholarship.

Amplifying research influence through the social network, open access publishing, and international collaboration: A mediation analysis on nursing studies literature

INTRODUCTION

Research impact and influence are commonly measured quantitatively by citation count received by research articles. Many institutes also use citation count as one of the factors in faculty performance appraisal and candidate selection of academic positions. Various strategies were recommended to amplify and accelerate research influence, particularly citation counts, by bringing research articles to a wider reach for potential readers. However, no prior empirical study was conducted to examine and valid effects of those strategies on nursing studies. This study examines and verifies the direct effects and mediation effects of some strategies, namely, the use of Twitter, international collaboration, the use of ResearchGate, and open access publishing, for amplifying the citation of research and review articles in nursing studies.

DESIGN

Cross-sectional study design.

METHODS

Articles published in top nursing journals in 2016 were identified in PUBMED and the citation metrics for individual articles until 2021 were extracted from Scopus. The primary outcome was the citation count of the article, while the tweet count on Twitter of the article was considered a mediator. The predictors included paper type, the total number of authors, the proportion of authors with a ResearchGate account in the article, funding support, open-accessed article, and the number of different countries stated in the authors' affiliation. A mediation analysis was conducted to examine the predictors' direct and indirect effects (i.e., via tweet count) on the citation count of the article.

RESULTS

A total of 2210 articles were included in this study, of which 223 (10.1%) were review articles. The median (IQR) number of Scopus citations, tweets, countries, and percentage of authors with ResearchGate accounts were 12 (6-21), 2 (0-6), 1 (1-1), and 75% (50%-100%) respectively. In the mediation analysis, tweet count, article type, number of countries, percentage of authors with a ResearchGate account, and journal impact factors in 2014 were positively associated with the Scopus citation count. The effects of article type, open access, and journals' impact factors in 2014 on Scopus citation count were mediated by the tweet count.

CONCLUSION

This study provides empirical support for some strategies researchers may employ to amplify the citation count of their research articles. The methodology of our study can be extended to compare research influence between entities (e.g., across countries or institutes).

CLINICAL RELEVANCE

The citation refers to the research work cited by peers and is one of the indicators for research impact. Higher citations implied the research work is read and used by others, therefore, understanding the associated factors with higher citations is critical.

Institutional dashboards on clinical trial transparency for University Medical Centers: A case study

BACKGROUND

University Medical Centers (UMCs) must do their part for clinical trial transparency by fostering practices such as prospective registration, timely results reporting, and open access. However, research institutions are often unaware of their performance on these practices. Baseline assessments of these practices would highlight where there is room for change and empower UMCs to support improvement. We performed a status quo analysis of established clinical trial registration and reporting practices at German UMCs and developed a dashboard to communicate these baseline assessments with UMC leadership and the wider research community.

METHODS AND FINDINGS

We developed and applied a semiautomated approach to assess adherence to established transparency practices in a cohort of interventional trials and associated results publications. Trials were registered in ClinicalTrials.gov or the German Clinical Trials Register (DRKS), led by a German UMC, and reported as complete between 2009 and 2017. To assess adherence to transparency practices, we identified results publications associated to trials and applied automated methods at the level of registry data (e.g., prospective registration) and publications (e.g., open access). We also obtained summary results reporting rates of due trials registered in the EU Clinical Trials Register (EUCTR) and conducted at German UMCs from the EU Trials Tracker. We developed an interactive dashboard to display these results across all UMCs and at the level of single UMCs. Our study included and assessed 2,895 interventional trials led by 35 German UMCs. Across all UMCs, prospective registration increased from 33% (n = 58/178) to 75% (n = 144/193) for trials registered in ClinicalTrials.gov and from 0% (n = 0/44) to 79% (n = 19/24) for trials registered in DRKS over the period considered. Of trials with a results publication, 38% (n = 714/1,895) reported the trial registration number in the publication abstract. In turn, 58% (n = 861/1,493) of trials registered in ClinicalTrials.gov and 23% (n = 111/474) of trials registered in DRKS linked the publication in the registration. In contrast to recent increases in summary results reporting of drug trials in the EUCTR, 8% (n = 191/2,253) and 3% (n = 20/642) of due trials registered in ClinicalTrials.gov and DRKS, respectively, had summary results in the registry. Across trial completion years, timely results reporting (within 2 years of trial completion) as a manuscript publication or as summary results was 41% (n = 1,198/2,892). The proportion of openly accessible trial publications steadily increased from 42% (n = 16/38) to 74% (n = 72/97) over the period considered. A limitation of this study is that some of the methods used to assess the transparency practices in this dashboard rely on registry data being accurate and up-to-date.

CONCLUSIONS

In this study, we observed that it is feasible to assess and inform individual UMCs on their performance on clinical trial transparency in a reproducible and publicly accessible way. Beyond helping institutions assess how they perform in relation to mandates or their institutional policy, the dashboard may inform interventions to increase the uptake of clinical transparency practices and serve to evaluate the impact of these interventions.

Ten simple rules for socially responsible science

Guidelines concerning the potentially harmful effects of scientific studies have historically focused on ethical considerations for minimizing risk for participants. However, studies can also indirectly inflict harm on individuals and social groups through how they are designed, reported, and disseminated. As evidenced by recent criticisms and retractions of high-profile studies dealing with a wide variety of social issues, there is a scarcity of resources and guidance on how one can conduct research in a socially responsible manner. As such, even motivated researchers might publish work that has negative social impacts due to a lack of awareness. To address this, we propose 10 simple rules for researchers who wish to conduct socially responsible science. These rules, which cover major considerations throughout the life cycle of a study from inception to dissemination, are not aimed as a prescriptive list or a deterministic code of conduct. Rather, they are meant to help motivated scientists to reflect on their social responsibility as researchers and actively engage with the potential social impact of their research.

Factors relating to nonpublication and publication bias in clinical trials in Canada: A qualitative interview study

AIMS

This study aims to understand factors contributing to nonpublication and publication bias in clinical trials in Canada.

METHODS

Qualitative interviews were conducted between March 2019 and April 2021 with 34 participants from the Canadian provinces of Alberta, British Columbia and Ontario, including 17 clinical trial investigators, 1 clinical research coordinator, 3 research administrators, 3 research ethics board members and 10 clinical trial participants. We conducted a thematic analysis involving coding of interview transcripts and memo-writing to identify key themes.

RESULTS

Several factors contribute to nonpublication and publication bias in clinical trial research. A core theme was that reporting practices are shaped by incentives within the research system taht favour publication of positive over negative trials. Investigators are discouraged from reporting by experiences or perceptions of difficulty in publishing negative findings but rewarded for publishing positive findings in various ways. Trial investigators more strongly associated positive clinical trials than negative trials with opportunities for industry and nonindustry funding and with academic promotion, bonuses and recognition. Research institutions and ethics boards tended to lack well-resourced, proactive policies and practices to ensure trial findings are reported in registries or journals.

CONCLUSION

Clinical trial reporting practices in Canada are shaped by incentives favouring reporting of positive over negative trials, such as funding opportunities and academic promotion, bonuses and recognition. Research institutions could help change incentives by adopting performance metrics that emphasize full reporting of results in journals or registries.

Healthy City Community Space-Oriented Structural Planning and Management Optimization under COVID-19

This work studies ways of Healthy City Construction (HCC) and Urban Governance Optimization (UGO) during the COVID-19 pandemic. The specific urban community space planning structure is proposed following a literature review on the healthy city's theoretical basis and historical development. Then, the proposed HCC-oriented community space structure is tested by surveying residents' physical and mental health and infectious risk using a questionnaire survey and Particle Swarm Optimization (PSO). Specifically, the particle fitness is calculated according to the original data conditions, and the community space with the highest fitness is determined. Based on the calculation, the community space's neighbors are investigated from different aspects through a questionnaire survey on patients' daily activities and community health security coverage. The results showed that: (1) The score of daily activities of community patients with respiratory diseases was 2312 before the implementation of the proposed community structure and 2715 after the implementation. Therefore, the service quality of residents increases after implementation. (2) The proposed HCC-oriented community space structure improves the physical self-control ability of chronic patients and helps them reduce their pain. This work aims to create a people-oriented healthy city community space, improve the city's "immune system," and regenerate the energy and environmental sustainability of the urban living environment.

DORA-compliant measures of research quality and impact to assess the performance of researchers in biomedical institutions: Review of published research, international best practice and Delphi survey

OBJECTIVE

The San Francisco Declaration on Research Assessment (DORA) advocates for assessing biomedical research quality and impact, yet academic organizations continue to employ traditional measures such as Journal Impact Factor. We aimed to identify and prioritize measures for assessing research quality and impact.

METHODS

We conducted a review of published and grey literature to identify measures of research quality and impact, which we included in an online survey. We assembled a panel of researchers and research leaders, and conducted a two-round Delphi survey to prioritize measures rated as high (rated 6 or 7 by ≥ 80% of respondents) or moderate (rated 6 or 7 by ≥ 50% of respondents) importance.

RESULTS

We identified 50 measures organized in 8 domains: relevance of the research program, challenges to research program, or productivity, team/open science, funding, innovations, publications, other dissemination, and impact. Rating of measures by 44 panelists (60%) in Round One and 24 (55%) in Round Two of a Delphi survey resulted in consensus on the high importance of 5 measures: research advances existing knowledge, research plan is innovative, an independent body of research (or fundamental role) supported by peer-reviewed research funding, research outputs relevant to discipline, and quality of the content of publications. Five measures achieved consensus on moderate importance: challenges to research productivity, potential to improve health or healthcare, team science, collaboration, and recognition by professional societies or academic bodies. There was high congruence between researchers and research leaders across disciplines.

CONCLUSIONS

Our work contributes to the field by identifying 10 DORA-compliant measures of research quality and impact, a more comprehensive and explicit set of measures than prior efforts. Research is needed to identify strategies to overcome barriers of use of DORA-compliant measures, and to "de-implement" traditional measures that do not uphold DORA principles yet are still in use.

The call for science communication and public scholarship

This commentary calls for a collective shift in the sciences and academia to prioritize and invest in communicating research in ways that are engaging, relevant, and accessible to public audiences. We provide the context and rationale for increasing and enhancing science communication, and identify barriers that prevent health researchers, educators, and practitioners from engaging with the public. Academic institutions need to develop and implement policies that encourage and support science communication and public scholarship initiatives that are sustainable and scalable.

Metrics to evaluate implementation scientists in the USA: what matters most?

Background

Implementation science has grown rapidly as a discipline over the past two decades. An examination of how publication patterns and other scholarly activities of implementation scientists are weighted in the tenure and promotion process is needed given the unique and applied focus of the field.

Methods

We surveyed implementation scientists (mostly from the USA) to understand their perspectives on the following matters: (1) factors weighted in tenure and promotion for implementation scientists, (2) how important these factors are for success as an implementation scientist, (3) how impact is defined for implementation scientists, (4) top journals in implementation science, and (5) how these journals are perceived with regard to their prestige. We calculated univariate descriptive statistics for all quantitative data, and we used Wilcoxon signed-rank tests to compare the participants’ ratings of various factors. We analyzed open-ended qualitative responses using content analysis.

Results

One hundred thirty-two implementation scientists completed the survey (response rate = 28.9%). Four factors were rated as more important for tenure and promotion decisions: number of publications, quality of publication outlets, success in obtaining external funding, and record of excellence in teaching. Six factors were rated as more important for overall success as an implementation scientist: presentations at professional meetings, involvement in professional service, impact of the implementation scientist’s scholarship on the local community and/or state, impact of the implementation scientist’s scholarship on the research community, the number and quality of the implementation scientist’s community partnerships, and the implementation scientist’s ability to disseminate their work to non-research audiences. Participants most frequently defined and described impact as changing practice and/or policy. This expert cohort identified Implementation Science as the top journal in the field.

Conclusions

Overall, there was a significant mismatch between the factors experts identified as being important to academic success (e.g., tenure and promotion) and the factors needed to be a successful implementation scientist. Findings have important implications for capacity building, although they are largely reflective of the promotion and tenure process in the USA.

Supplementary Information

The online version contains supplementary material available at 10.1186/s43058-022-00323-0.

Rigor and reproducibility for data analysis and design in the study of eating disorders

OBJECTIVE

Incorporating open science practices has become a priority for submission criteria in the International Journal of Eating Disorders (IJED). In this systematic review, we used the rigor and reproducibility framework developed by Hildebrandt and Prenoveau (2020) to examine the implementation of statistically sound open science principles in IJED, determining whether the cost and effort of incorporating these practices ultimately make research more likely to be cited.

METHOD

For this systematic review, six trained coders examined 1145 articles published from January 2011 to May 2021, including the 5 years prior to the 2016 introduction of the Open Science Foundation article preregistration. We coded for the presence or absence of 10 specific open science elements and calculated citation metrics for each article.

RESULTS

There was evidence of a significant positive relationship between time and total rigor and reproducibility (Total RR) criteria included in IJED articles following the implementation of preregistration in 2016. For every increase in year from 2011 to 2016, there was a .14 decrease in Total RR criteria. From 2016 to 2021, there was a .42 increase per volume in Total RR criteria. There was no statistically significant relationship between Total RR criteria and citation impact.

DISCUSSION

Although findings indicate that statistical rigor and reproducibility in this field has increased, the lack of direct relationship between open science methods and article visibility for scientists suggests that there is a limited incentive for researchers to participate in reporting guidelines.

PUBLIC SIGNIFICANCE

Statistical controversies within science threaten the rigor and reproducibility of published research. Open science practices, including the preregistration of study hypotheses, links to statistical code, and explicit data-sharing arguably generate reliable and valid inferences. This review illustrates the rigor and reproducibility of articles published in IJED between 2011 and 2021 and identifies whether open sciences practices have become increasingly prevalent in eating disorder research.

The failure of success: four lessons learned in five years of research on research integrity and research assessments

In the past 5 years, we captured the perspectives from a broad array of research stakeholders to better understand the impact that current approaches to success and research assessment may have on the integrity and the quality of research. Here, we translate our findings in four actions that are urgently needed to foster better research. First, we need to address core research structures to overcome systemic problems of the research enterprise; second, we must realign research assessments to value elements that advance and strengthen science; third, we need to remodel, diversify, and secure research careers; and finally, we need to unite and coordinate efforts for change.

Untangling the network effects of productivity and prominence among scientists

While inequalities in science are common, most efforts to understand them treat scientists as isolated individuals, ignoring the network effects of collaboration. Here, we develop models that untangle the network effects of productivity defined as paper counts, and prominence referring to high-impact publications, of individual scientists from their collaboration networks. We find that gendered differences in the productivity and prominence of mid-career researchers can be largely explained by differences in their coauthorship networks. Hence, collaboration networks act as a form of social capital, and we find evidence of their transferability from senior to junior collaborators, with benefits that decay as researchers age. Collaboration network effects can also explain a large proportion of the productivity and prominence advantages held by researchers at prestigious institutions. These results highlight a substantial role of social networks in driving inequalities in science, and suggest that collaboration networks represent an important form of unequally distributed social capital that shapes who makes what scientific discoveries.

While inequalities in science are common, most efforts to understand them treat scientists as isolated individuals, ignoring the network effects of collaboration. Here, the authors develop models that untangle the network effects of productivity and prominence of individual scientists from their collaboration networks.

Citation counts and journal impact factors do not capture some indicators of research quality in the behavioural and brain sciences

Citation data and journal impact factors are important components of faculty dossiers and figure prominently in both promotion decisions and assessments of a researcher's broader societal impact. Although these metrics play a large role in high-stakes decisions, the evidence is mixed about whether they are strongly correlated with indicators of research quality. We use data from a large-scale dataset comprising 45 144 journal articles with 667 208 statistical tests and data from 190 replication attempts to assess whether citation counts and impact factors predict three indicators of research quality: (i) the accuracy of statistical reporting, (ii) the evidential value of the reported data and (iii) the replicability of a given experimental result. Both citation counts and impact factors were weak and inconsistent predictors of research quality, so defined, and sometimes negatively related to quality. Our findings raise the possibility that citation data and impact factors may be of limited utility in evaluating scientists and their research. We discuss the implications of these findings in light of current incentive structures and discuss alternative approaches to evaluating research.

The impact of Italian performance-based research funding systems on the intensity of international research collaboration

The study of national research assessment exercises serves to evaluate the effectiveness of policies versus their objectives and to improve the formulation of future initiatives. The aim of the current study is to verify whether the introduction of the first performance-based research funding in Italy, based on the 2004–10 VQR assessment, achieved the intended objective of inducing greater international collaboration on the part of researchers. For this, we apply a bibliometric approach based on the observation of coauthorships in Italian and worldwide scientific publications over a 14-year period, beginning in the target years of the VQR assessment. Through an Interrupted Time Series Analysis, we compare the expected and observed patterns of international coauthorship for Italy and the rest of the world. Although the rate of internationalization of Italian research is seen to be increasing, whether this is a consequence of the VQR incentives, or rather part of a global phenomenon of recourse to international collaboration in response to the increasingly complex scientific challenges, it is open to interpretation.

Stakeholders’ views on an institutional dashboard with metrics for responsible research

Concerns about research waste have fueled debate about incentivizing individual researchers and research institutions to conduct responsible research. We showed stakeholders a proof-of-principle dashboard with quantitative metrics of responsible research practices at University Medical Centers (UMCs). Our research question was: What are stakeholders’ views on a dashboard that displays the adoption of responsible research practices on a UMC-level? We recruited stakeholders (UMC leadership, support staff, funders, and experts in responsible research) to participate in online interviews. We applied content analysis to understand what stakeholders considered the strengths, weaknesses, opportunities, and threats of the dashboard and its metrics. Twenty-eight international stakeholders participated in online interviews. Stakeholders considered the dashboard helpful in providing a baseline before designing interventions and appreciated the focus on concrete behaviors. Main weaknesses concerned the lack of an overall narrative justifying the choice of metrics. Stakeholders hoped the dashboard would be supplemented with other metrics in the future but feared that making the dashboard public might put UMCs in a bad light. Our findings furthermore suggest a need for discussion with stakeholders to develop an overarching framework for responsible research evaluation and to get research institutions on board.

Closing the knowledge-action gap in conservation with open science

The knowledge-action gap in conservation science and practice occurs when research outputs do not result in actions to protect or restore biodiversity. Among the diverse and complex reasons for this gap, three barriers are fundamental: knowledge is often unavailable to practitioners and challenging to interpret or difficult to use or both. Problems of availability, interpretability, and useability are solvable with open science practices. We considered the benefits and challenges of three open science practices for use by conservation scientists and practitioners. First, open access publishing makes the scientific literature available to all. Second, open materials (detailed methods, data, code, and software) increase the transparency and use of research findings. Third, open education resources allow conservation scientists and practitioners to acquire the skills needed to use research outputs. The long-term adoption of open science practices would help researchers and practitioners achieve conservation goals more quickly and efficiently and reduce inequities in information sharing. However, short-term costs for individual researchers (insufficient institutional incentives to engage in open science and knowledge mobilization) remain a challenge. We caution against a passive approach to sharing that simply involves making information available. We advocate a proactive stance toward transparency, communication, collaboration, and capacity building that involves seeking out and engaging with potential users to maximize the environmental and societal impact of conservation science.

Publication by association: how the COVID-19 pandemic has shown relationships between authors and editorial board members in the field of infectious diseases

During the COVID-19 pandemic, the rush to scientific and political judgements on the merits of hydroxychloroquine was fuelled by dubious papers which may have been published because the authors were not independent from the practices of the journals in which they appeared. This example leads us to consider a new type of illegitimate publishing entity, 'self-promotion journals' which could be deployed to serve the instrumentalisation of productivity-based metrics, with a ripple effect on decisions about promotion, tenure and grant funding, but also on the quality of manuscripts that are disseminated to the medical community and form the foundation of evidence-based medicine.