Academic criteria for promotion and tenure in biomedical sciences faculties: cross sectional analysis of international sample of universities

Relationship of adaptive expertise of health professions educators with age, experience, academic rank, and their work performance during an altered academic environment

BACKGROUND

Health professionals and health professions educators (HPEs) worldwide were confronted by the COVID-19 pandemic, which disrupted standard practice and forced HPEs to develop creative, alternative modes of training and education. The ability of people to work successfully and efficiently in non-standard situations can be called adaptive expertise in which people quickly overcome changes in work requirements using their expert knowledge in novel ways. The objectives of the current study were to investigate how the adaptive expertise of a group of HPEs influenced perceived work performance in a non-standard situation and to see whether there were relationships between the level of adaptive expertise and academic ranking and work experience of HPEs.

METHODS

A descriptive, cross-sectional, single-site study was conducted using a self-reported study tool about adaptive expertise developed by Carbonell et al. (2016), and three questions were asked about participants' perceptions of work performance, amount of work done, and teaching quality. The sample consisted of HPEs from the University of Twente, Netherlands.

RESULTS

Among 123 eligible participants, 40 individuals completed the survey. Kaiser-Meyer-Olkin and Bartlett's Test of Sphericity indicated the adequacy of the sample size (KMO = 0.633, P < 0.0001). Participants were lecturers, senior lecturers, assistant professors, associate professors and full professors. The average adaptive expertise score of the sample was 4.18 ± 0.57 on a scale from 1 (low) to 5 (high). The domain and innovative skills are the principal distinct dimensions of adaptive expertise among HPEs. Professors showed higher adaptive expertise scores than the other ranks. Statistically significant correlations were found between scores of adaptive expertise and perceived work performance (r = 0.41, p < 0.05 and academic ranking (r = 0.42, p < 0.05). Adaptive expertise scores were not associated with work experience or HPEs' age.

CONCLUSIONS

Our finding of a lack of relationships between self-reported level of adaptive expertise and experience and age but significant relationships with work performance and academic ranking of HPEs suggests that adaptive expertise is not auto-generated or acquired with seniority and experience but is a 'mastery' that should be developed deliberately.

Recognising the importance and impact of Imaging Scientists: Global guidelines for establishing career paths within core facilities

In the dynamic landscape of scientific research, imaging core facilities are vital hubs propelling collaboration and innovation at the technology development and dissemination frontier. Here, we present a collaborative effort led by Global BioImaging (GBI), introducing international recommendations geared towards elevating the careers of Imaging Scientists in core facilities. Despite the critical role of Imaging Scientists in modern research ecosystems, challenges persist in recognising their value, aligning performance metrics and providing avenues for career progression and job security. The challenges encompass a mismatch between classic academic career paths and service-oriented roles, resulting in a lack of understanding regarding the value and impact of Imaging Scientists and core facilities and how to evaluate them properly. They further include challenges around sustainability, dedicated training opportunities and the recruitment and retention of talent. Structured across these interrelated sections, the recommendations within this publication aim to propose globally applicable solutions to navigate these challenges. These recommendations apply equally to colleagues working in other core facilities and research institutions through which access to technologies is facilitated and supported. This publication emphasises the pivotal role of Imaging Scientists in advancing research programs and presents a blueprint for fostering their career progression within institutions all around the world.

Mind the Gap! Working Toward Gender Equity in Pediatric and Congenital Heart Disease: Present and Future

Evidence from medicine and other fields has shown that gender diversity results in better decision making and outcomes. The incoming workforce of congenital heart specialists (especially in pediatric cardiology) appears to be more gender balanced, but past studies have shown many inequities. Gender-associated differences in leadership positions, opportunities presented for academic advancement, and recognition for academic contributions to the field persist. In addition, compensation packages remain disparate if evaluated based on gender with equivalent experience and expertise. This review explores these inequities and has suggested individual and institutional changes that could be made to recruit and retain women, monitor the climate of the institution, and identify and eliminate bias in areas like salary and promotions.

Increasing Value and Reducing Waste of Research on Neurofeedback Effects in Post-traumatic Stress Disorder: A State-of-the-Art-Review

Post-Traumatic Stress Disorder (PTSD) is often considered challenging to treat due to factors that contribute to its complexity. In the last decade, more attention has been paid to non-pharmacological or non-psychological therapies for PTSD, including neurofeedback (NFB). NFB is a promising non-invasive technique targeting specific brainwave patterns associated with psychiatric symptomatology. By learning to regulate brain activity in a closed-loop paradigm, individuals can improve their functionality while reducing symptom severity. However, owing to its lax regulation and heterogeneous legal status across different countries, the degree to which it has scientific support as a psychiatric treatment remains controversial. In this state-of-the-art review, we searched PubMed, Cochrane Central, Web of Science, Scopus, and MEDLINE and identified meta-analyses and systematic reviews exploring the efficacy of NFB for PTSD. We included seven systematic reviews, out of which three included meta-analyses (32 studies and 669 participants) that targeted NFB as an intervention while addressing a single condition-PTSD. We used the MeaSurement Tool to Assess systematic Reviews (AMSTAR) 2 and the criteria described by Cristea and Naudet (Behav Res Therapy 123:103479, 2019, https://doi.org/10.1016/j.brat.2019.103479 ) to identify sources of research waste and increasing value in biomedical research. The seven assessed reviews had an overall extremely poor quality score (5 critically low, one low, one moderate, and none high) and multiple sources of waste while opening opportunities for increasing value in the NFB literature. Our research shows that it remains unclear whether NFB training is significantly beneficial in treating PTSD. The quality of the investigated literature is low and maintains a persistent uncertainty over numerous points, which are highly important for deciding whether an intervention has clinical efficacy. Just as importantly, none of the reviews we appraised explored the statistical power, referred to open data of the included studies, or adjusted their pooled effect sizes for publication bias and risk of bias. Based on the obtained results, we identified some recurrent sources of waste (such as a lack of research decisions based on sound questions or using an appropriate methodology in a fully transparent, unbiased, and useable manner) and proposed some directions for increasing value (homogeneity and consensus) in designing and reporting research on NFB interventions in PTSD.

Research rigor and reproducibility in research education: A CTSA institutional survey

We assessed the rigor and reproducibility (R&R) activities of institutions funded by the National Center for Advancing Translational Sciences (NCTSA) through a survey and website search (N = 61). Of 50 institutional responses, 84% reported incorporating some form of R&R training, 68% reported devoted R&R training, 30% monitored R&R practices, and 10% incentivized them. Website searches revealed 9 (15%) freely available training curricula, and 7 (11%) institutional programs specifically created to enhance R&R. NCATS should formally integrate R&R principles into its translational science models and institutional requirements.

Authorship inequalities in global health research: the IeDEA Southern Africa collaboration

BACKGROUND

The International epidemiology Databases to Evaluate AIDS conducts research in several regions, including in Southern Africa. We assessed authorship inequalities for the Southern African region, which is led by South African and Swiss investigators.

METHODS

We analysed authorships of publications from 2007 to 2020 by gender, country income group, time and citation impact. We used 2020 World Bank categories to define income groups and the relative citation ratio (RCR) to assess citation impact. Authorship parasitism was defined as articles without authors from the countries where the study was conducted. A regression model examined the probability of different authorship positions.

RESULTS

We included 313 articles. Of the 1064 contributing authors, 547 (51.4%) were women, and 223 (21.0%) were from 32 low-income/lower middle-income countries (LLMICs), 269 (25.3%) were from 13 upper middle-income countries and 572 (53.8%) were from 25 high-income countries (HICs). Most articles (150/157, 95.5%) reporting data from Southern Africa included authors from all participating countries. Women were more likely to be the first author than men (OR 1.74; 95% CI 1.06 to 2.83) but less likely to be last authors (OR 0.63; 95% CI 0.40 to 0.99). Compared with HIC, LLMIC authors were less likely to publish as first (OR 0.21; 95% CI 0.11 to 0.41) or last author (OR 0.20; 95% CI 0.09 to 0.42). The proportion of women and LLMIC first and last authors increased over time. The RCR tended to be higher, indicating greater impact, if first or last authors were from HIC (p=0.06).

CONCLUSIONS

This analysis of a global health collaboration co-led by South African and Swiss investigators showed little evidence of authorship parasitism. There were stark inequalities in authorship position, with women occupying more first and men more last author positions and researchers from LLMIC being 'stuck in the middle' on the byline. Global health research collaborations should monitor, analyse and address authorship inequalities.

Early-career factors largely determine the future impact of prominent researchers: evidence across eight scientific fields

Can we help predict the future impact of researchers using early-career factors? We analyze early-career factors of the world's 100 most prominent researchers across 8 scientific fields and identify four key drivers in researchers' initial career: working at a top 25 ranked university, publishing a paper in a top 5 ranked journal, publishing most papers in top quartile (high-impact) journals and co-authoring with other prominent researchers in their field. We find that over 95% of prominent researchers across multiple fields had at least one of these four features in the first 5 years of their career. We find that the most prominent scientists who had an early career advantage in terms of citations and h-index are more likely to have had all four features, and that this advantage persists throughout their career after 10, 15 and 20 years. Our findings show that these few early-career factors help predict researchers' impact later in their careers. Our research thus points to the need to enhance fairness and career mobility among scientists who have not had a jump start early on.

Evaluation of the promotion criteria in an academic medical centre in Singapore

INTRODUCTION

Academic medical centres (AMCs) have the tripartite mission of performing research to advance healthcare delivery, educating future clinicians and providing healthcare services. This study investigates the criteria associated with being promoted in a Singaporean AMC.

METHODS

Using a dataset of 255 candidates for promotion at the studied AMC, we employ logistic regression to determine if these factors are associated with the likelihood of promotion. Further, we use interaction effects to test if the relationship between the H-index and likelihood of promotion differs across the academic levels of the candidates.

RESULTS

The logistic regression results based on the best of our three tested models suggest that the H-index is positively associated with promotion for those applying to become clinical associate professors (OR=1.43, p=0.01). Moreover, candidates who provide well-developed education portfolios (OR=3.61, p=0.02) and who have held service/leadership roles (OR=6.72, p<0.001) are more likely to be promoted.

CONCLUSIONS

This study affirms the correlation between promotion and the advancement criteria outlined by the AMC. This is important for transparency and trust between the AMC and its faculty in their applications for promotion and success in an academic career. Further, our study is one of the few empirical studies linking promotion criteria to promotion outcomes.

Open Science 2.0: Towards a truly collaborative research ecosystem

Conversations about open science have reached the mainstream, yet many open science practices such as data sharing remain uncommon. Our efforts towards openness therefore need to increase in scale and aim for a more ambitious target. We need an ecosystem not only where research outputs are openly shared but also in which transparency permeates the research process from the start and lends itself to more rigorous and collaborative research. To support this vision, this Essay provides an overview of a selection of open science initiatives from the past 2 decades, focusing on methods transparency, scholarly communication, team science, and research culture, and speculates about what the future of open science could look like. It then draws on these examples to provide recommendations for how funders, institutions, journals, regulators, and other stakeholders can create an environment that is ripe for improvement.

Authorship and citation patterns of highly cited biomedical researchers: a cross-sectional study

BACKGROUND

Scientific productivity is often evaluated by means of cumulative citation metrics. Different metrics produce different incentives. The H-index assigns full credit from a citation to each coauthor, and thus may encourage multiple collaborations in mid-list author roles. In contrast, the Hm-index assigns only a fraction 1/k of citation credit to each of k coauthors of an article, and thus may encourage research done by smaller teams, and in first or last author roles. Whether H and Hm indices are influenced by different authorship patterns has not been examined.

METHODS

Using a publicly available Scopus database, I examined associations between the numbers of research articles published as single, first, mid-list, or last author between 1990 and 2019, and the H-index and the Hm-index, among 18,231 leading researchers in the health sciences.

RESULTS

Adjusting for career duration and other article types, the H-index was negatively associated with the number of single author articles (partial Pearson r -0.06) and first author articles (-0.08), but positively associated with the number of mid-list (0.64) and last author articles (0.21). In contrast, all associations were positive for the Hm-index (0.04 for single author articles, 0.18 for first author articles, 0.24 for mid-list articles, and 0.46 for last author articles).

CONCLUSION

The H-index and the Hm-index do not reflect the same authorship patterns: the full-credit H-index is predominantly associated with mid-list authorship, whereas the partial-credit Hm-index is driven by more balanced publication patterns, and is most strongly associated with last-author articles. Since performance metrics may act as incentives, the selection of a citation metric should receive careful consideration.

Viewing research assessment, the academic reward system, and academic publishing through the power/knowledge lens of Foucault

The academic research assessment system, the academic reward system, and the academic publishing system are interrelated mechanisms that facilitate the scholarly production of knowledge. This article considers these systems using a Foucauldian lens to examine the power/knowledge relationships found within and through these systems. A brief description of the various systems is introduced followed by examples of instances where Foucault's power, knowledge, discourse, and power/knowledge concepts are useful to provide a broader understanding of the norms and rules associated with each system, how these systems form a network of power relationships that reinforce and shape one another.

Impact of major awards on the subsequent work of their recipients

To characterize the impact of major research awards on recipients' subsequent work, we studied Nobel Prize winners in Chemistry, Physiology or Medicine, and Physics and MacArthur Fellows working in scientific fields. Using a case-crossover design, we compared scientists' citations, publications and citations-per-publication from work published in a 3-year pre-award period to their work published in a 3-year post-award period. Nobel Laureates and MacArthur Fellows received fewer citations for post- than for pre-award work. This was driven mostly by Nobel Laureates. Median decrease was 80.5 citations among Nobel Laureates (p = 0.004) and 2 among MacArthur Fellows (p = 0.857). Mid-career (42-57 years) and senior (greater than 57 years) researchers tended to earn fewer citations for post-award work. Early career researchers (less than 42 years, typically MacArthur Fellows) tended to earn more, but the difference was non-significant. MacArthur Fellows (p = 0.001) but not Nobel Laureates (p = 0.180) had significantly more post-award publications. Both populations had significantly fewer post-award citations per paper (p = 0.043 for Nobel Laureates, 0.005 for MacArthur Fellows, and 0.0004 for combined population). If major research awards indeed fail to increase (and even decrease) recipients' impact, one may need to reassess the purposes, criteria, and impacts of awards to improve the scientific enterprise.

Responsible research practices could be more strongly endorsed by Australian university codes of research conduct

BACKGROUND

This study aimed to investigate how strongly Australian university codes of research conduct endorse responsible research practices.

METHODS

Codes of research conduct from 25 Australian universities active in health and medical research were obtained from public websites, and audited against 19 questions to assess how strongly they (1) defined research integrity, research quality, and research misconduct, (2) required research to be approved by an appropriate ethics committee, (3) endorsed 9 responsible research practices, and (4) discouraged 5 questionable research practices.

RESULTS

Overall, a median of 10 (IQR 9 to 12) of 19 practices covered in the questions were mentioned, weakly endorsed, or strongly endorsed. Five to 8 of 9 responsible research practices were mentioned, weakly, or strongly endorsed, and 3 questionable research practices were discouraged. Results are stratified by Group of Eight (n = 8) and other (n = 17) universities. Specifically, (1) 6 (75%) Group of Eight and 11 (65%) other codes of research conduct defined research integrity, 4 (50%) and 8 (47%) defined research quality, and 7 (88%) and 16 (94%) defined research misconduct. (2) All codes required ethics approval for human and animal research. (3) All codes required conflicts of interest to be declared, but there was variability in how strongly other research practices were endorsed. The most commonly endorsed practices were ensuring researcher training in research integrity [8 (100%) and 16 (94%)] and making study data publicly available [6 (75%) and 12 (71%)]. The least commonly endorsed practices were making analysis code publicly available [0 (0%) and 0 (0%)] and registering analysis protocols [0 (0%) and 1 (6%)]. (4) Most codes discouraged fabricating data [5 (63%) and 15 (88%)], selectively deleting or modifying data [5 (63%) and 15 (88%)], and selective reporting of results [3 (38%) and 15 (88%)]. No codes discouraged p-hacking or hypothesising after results are known.

CONCLUSIONS

Responsible research practices could be more strongly endorsed by Australian university codes of research conduct. Our findings may not be generalisable to smaller universities, or those not active in health and medical research.

Exploring the merits of research performance measures that comply with the San Francisco Declaration on Research Assessment and strategies to overcome barriers of adoption: qualitative interviews with administrators and researchers

BACKGROUND

In prior research, we identified and prioritized ten measures to assess research performance that comply with the San Francisco Declaration on Research Assessment, a principle adopted worldwide that discourages metrics-based assessment. Given the shift away from assessment based on Journal Impact Factor, we explored potential barriers to implementing and adopting the prioritized measures.

METHODS

We identified administrators and researchers across six research institutes, conducted telephone interviews with consenting participants, and used qualitative description and inductive content analysis to derive themes.

RESULTS

We interviewed 18 participants: 6 administrators (research institute business managers and directors) and 12 researchers (7 on appointment committees) who varied by career stage (2 early, 5 mid, 5 late). Participants appreciated that the measures were similar to those currently in use, comprehensive, relevant across disciplines, and generated using a rigorous process. They also said the reporting template was easy to understand and use. In contrast, a few administrators thought the measures were not relevant across disciplines. A few participants said it would be time-consuming and difficult to prepare narratives when reporting the measures, and several thought that it would be difficult to objectively evaluate researchers from a different discipline without considerable effort to read their work. Strategies viewed as necessary to overcome barriers and support implementation of the measures included high-level endorsement of the measures, an official launch accompanied by a multi-pronged communication strategy, training for both researchers and evaluators, administrative support or automated reporting for researchers, guidance for evaluators, and sharing of approaches across research institutes.

CONCLUSIONS

While participants identified many strengths of the measures, they also identified a few limitations and offered corresponding strategies to address the barriers that we will apply at our organization. Ongoing work is needed to develop a framework to help evaluators translate the measures into an overall assessment. Given little prior research that identified research assessment measures and strategies to support adoption of those measures, this research may be of interest to other organizations that assess the quality and impact of research.

Gender differences in peer reviewed grant applications, awards, and amounts: a systematic review and meta-analysis

BACKGROUND

Differential participation and success in grant applications may contribute to women's lesser representation in the sciences. This study's objective was to conduct a systematic review and meta-analysis to address the question of gender differences in grant award acceptance rates and reapplication award acceptance rates (potential bias in peer review outcomes) and other grant outcomes.

METHODS

The review was registered on PROSPERO (CRD42021232153) and conducted in accordance with PRISMA 2020 standards. We searched Academic Search Complete, PubMed, and Web of Science for the timeframe 1 January 2005 to 31 December 2020, and forward and backward citations. Studies were included that reported data, by gender, on any of the following: grant applications or reapplications, awards, award amounts, award acceptance rates, or reapplication award acceptance rates. Studies that duplicated data reported in another study were excluded. Gender differences were investigated by meta-analyses and generalized linear mixed models. Doi plots and LFK indices were used to assess reporting bias.

RESULTS

The searches identified 199 records, of which 13 were eligible. An additional 42 sources from forward and backward searches were eligible, for a total of 55 sources with data on one or more outcomes. The data from these studies ranged from 1975 to 2020: 49 sources were published papers and six were funders' reports (the latter were identified by forwards and backwards searches). Twenty-nine studies reported person-level data, 25 reported application-level data, and one study reported both: person-level data were used in analyses. Award acceptance rates were 1% higher for men, which was not significantly different from women (95% CI 3% more for men to 1% more for women, k = 36, n = 303,795 awards and 1,277,442 applications, I² = 84%). Reapplication award acceptance rates were significantly higher for men (9%, 95% CI 18% to 1%, k = 7, n = 7319 applications and 3324 awards, I² = 63%). Women received smaller award amounts (g = -2.28, 95% CI -4.92 to 0.36, k = 13, n = 212,935, I² = 100%).

CONCLUSIONS

The proportions of women that applied for grants, re-applied, accepted awards, and accepted awards after reapplication were less than the proportion of eligible women. However, the award acceptance rate was similar for women and men, implying no gender bias in this peer reviewed grant outcome. Women received smaller awards and fewer awards after re-applying, which may negatively affect continued scientific productivity. Greater transparency is needed to monitor and verify these data globally.

Assessing the publishing priorities and preferences among STEM researchers at a large R1 institution

The cost of academic publishing has increased substantially despite the ease with which information can be shared on the web. Open Access publishing is a key mechanism for amplifying research access, inclusivity, and impact. Despite this, shifting to a free-to-read publishing environment requires navigating complex barriers that vary by career status and publishing expectations. In this article, we investigate the motivations and preferences of researchers situated within our large research institution as a case study for publishing attitudes at similar institutions. We surveyed the publishing priorities and preferences of researchers at various career stages in STEM fields as they relate to openness, data practices, and assessment of research impact. Our results indicate that publishing preferences, data management experience and research impact assessment vary by career status and departmental approaches to promotion. We find that open access publishing is widely appreciated regardless of career status, but financial limitations and publishing expectations were common barriers to publishing in Open Access journals. Our findings shed light on publishing attitudes and preferences among researchers at a major R1 research institution, and offer insight into advocacy strategies that incentivize open access publishing.

Factors relating to nonpublication and publication bias in clinical trials in Canada: A qualitative interview study

AIMS

This study aims to understand factors contributing to nonpublication and publication bias in clinical trials in Canada.

METHODS

Qualitative interviews were conducted between March 2019 and April 2021 with 34 participants from the Canadian provinces of Alberta, British Columbia and Ontario, including 17 clinical trial investigators, 1 clinical research coordinator, 3 research administrators, 3 research ethics board members and 10 clinical trial participants. We conducted a thematic analysis involving coding of interview transcripts and memo-writing to identify key themes.

RESULTS

Several factors contribute to nonpublication and publication bias in clinical trial research. A core theme was that reporting practices are shaped by incentives within the research system taht favour publication of positive over negative trials. Investigators are discouraged from reporting by experiences or perceptions of difficulty in publishing negative findings but rewarded for publishing positive findings in various ways. Trial investigators more strongly associated positive clinical trials than negative trials with opportunities for industry and nonindustry funding and with academic promotion, bonuses and recognition. Research institutions and ethics boards tended to lack well-resourced, proactive policies and practices to ensure trial findings are reported in registries or journals.

CONCLUSION

Clinical trial reporting practices in Canada are shaped by incentives favouring reporting of positive over negative trials, such as funding opportunities and academic promotion, bonuses and recognition. Research institutions could help change incentives by adopting performance metrics that emphasize full reporting of results in journals or registries.

Team science criteria and processes for promotion and tenure of Health Science University Faculty

Although team science has expanded with far-reaching benefits, universities generally have not established criteria to recognize its value in faculty promotion and tenure. This paper recommends how institutions might weigh a faculty member's engagement in team science in the promotion and tenure process. Seventeen team science promotion and tenure criteria are recommended based on four sources - an evaluation framework, effectiveness metrics, collaborative influences, and authorship criteria. Suggestions are made for adaptation of the 17 criteria to committee guidelines, faculty team science portfolios, and the roles of individuals and institutions participating in large, cross-disciplinary research projects. Future research recommendations are advanced.

Toward equitable open research: stakeholder co-created recommendations for research institutions, funders and researchers

Open Research aims to make research more accessible, transparent, reproducible, shared and collaborative. Doing so is meant to democratize and diversify access to knowledge and knowledge production, and ensure that research is useful outside of academic contexts. Increasing equity is therefore a key aim of the Open Research movement, yet mounting evidence demonstrates that the practices of Open Research are implemented in ways that undermine this. In response, we convened a diverse community of researchers, research managers and funders to co-create actionable recommendations for supporting the equitable implementation of Open Research. Using a co-creative modified Delphi method, we generated consensus-driven recommendations that address three key problem areas: the resource-intensive nature of Open Research, the high cost of article processing charges, and obstructive reward and recognition practices at funders and research institutions that undermine the implementation of Open Research. In this paper, we provide an overview of these issues, a detailed description of the co-creative process, and present the recommendations and the debates that surrounded them. We discuss these recommendations in relation to other recently published ones and conclude that implementing ours requires 'global thinking' to ensure that a systemic and inclusive approach to change is taken.

Language Barriers in Organismal Biology: What Can Journals Do Better?

In the field of organismal biology, as in much of academia, there is a strong incentive to publish in internationally recognized, highly regarded, English-language journals to promote career advancement. This expectation has created a linguistic hegemony in scientific publishing, whereby scholars for whom English is an additional language face additional barriers to achieving the same scientific recognition as scholars who speak English as a first language. Here, we surveyed the author guidelines of 230 journals in organismal biology with impact factors of 1.5 or greater for linguistically inclusive and equitable practices and policies. We looked for efforts that reflect first steps toward reducing barriers to publication for authors globally, including the presence of statements that encouraged submissions from authors of diverse nationalities and backgrounds, policies regarding manuscript rejection based on perceived inadequacies of the English language, the existence of bias-conscious reviewer practices, whether translation and editing resources or services are available, allowance for non-English abstracts, summaries, or translations, and whether journals offer license options that would permit authors (or other scholars) to translate their work and publish it elsewhere. We also directly contacted a subset of journals to verify whether the information on their author guidelines page accurately reflects their policies and the accommodations they would make. We reveal that journals and publishers have made little progress toward beginning to recognize or reduce language barriers. Counter to our predictions, journals associated with scientific societies did not appear to have more inclusive policies compared to non-society journals. Many policies lacked transparency and clarity, which can generate uncertainty, result in avoidable manuscript rejections, and necessitate additional time and effort from both prospective authors and journal editors. We highlight examples of equitable policies and summarize actions that journals can take to begin to alleviate barriers to scientific publishing.

Gender disparity in academic advancement: exploring differences among adult and pediatric radiologists

BACKGROUND

Gender imbalance in research output and academic rank in academic radiology is well-documented and long-standing. Less is known regarding this imbalance among pediatric radiologists.

OBJECTIVE

To characterize gender differences for academic rank and scholarly productivity of pediatric radiologists relative to adult radiologists.

MATERIALS AND METHODS

During summer 2021, faculty data for the top 10 U.S. News & World Report ranked adult radiology programs and the top 12 largest pediatric hospital radiology departments were collected. Information regarding self-reported gender, age, years of practice and academic rank was accessed from institutional websites and public provider databases. The h-index and the number of publications were acquired via Scopus. Group comparisons were performed using Mann-Whitney and chi-square tests.

RESULTS

Three hundred and sixty-four (160 women) pediatric and 1,170 (468 women) adult radiologists were included. Compared to adult radiologists, there were significantly fewer pediatric radiologists in advanced ranks (associate or full professor) (P = 0.024), driven by differences between male (P = 0.033) but not female radiologists (P = 0.67). Among pediatric radiologists, there was no significant difference in years in practice (P = 0.29) between males and females. There also was no significant difference in academic rank by gender (P = 0.37), different from adult radiology where men outnumber women in advanced ranks (P < 0.001). Male pediatric radiologists displayed higher academic productivity (h-index: 9.0 vs. 7.0; P = 0.01 and number of publications: 31 vs. 18; P = 0.003) than their female colleagues.

CONCLUSION

Academic pediatric radiology seems to have more equitable academic advancement than academic adult radiology. Despite similar time in the workforce, academic output among female pediatric radiologists lags that of their male colleagues.

DORA-compliant measures of research quality and impact to assess the performance of researchers in biomedical institutions: Review of published research, international best practice and Delphi survey

OBJECTIVE

The San Francisco Declaration on Research Assessment (DORA) advocates for assessing biomedical research quality and impact, yet academic organizations continue to employ traditional measures such as Journal Impact Factor. We aimed to identify and prioritize measures for assessing research quality and impact.

METHODS

We conducted a review of published and grey literature to identify measures of research quality and impact, which we included in an online survey. We assembled a panel of researchers and research leaders, and conducted a two-round Delphi survey to prioritize measures rated as high (rated 6 or 7 by ≥ 80% of respondents) or moderate (rated 6 or 7 by ≥ 50% of respondents) importance.

RESULTS

We identified 50 measures organized in 8 domains: relevance of the research program, challenges to research program, or productivity, team/open science, funding, innovations, publications, other dissemination, and impact. Rating of measures by 44 panelists (60%) in Round One and 24 (55%) in Round Two of a Delphi survey resulted in consensus on the high importance of 5 measures: research advances existing knowledge, research plan is innovative, an independent body of research (or fundamental role) supported by peer-reviewed research funding, research outputs relevant to discipline, and quality of the content of publications. Five measures achieved consensus on moderate importance: challenges to research productivity, potential to improve health or healthcare, team science, collaboration, and recognition by professional societies or academic bodies. There was high congruence between researchers and research leaders across disciplines.

CONCLUSIONS

Our work contributes to the field by identifying 10 DORA-compliant measures of research quality and impact, a more comprehensive and explicit set of measures than prior efforts. Research is needed to identify strategies to overcome barriers of use of DORA-compliant measures, and to "de-implement" traditional measures that do not uphold DORA principles yet are still in use.

Metrics to evaluate implementation scientists in the USA: what matters most?

Background

Implementation science has grown rapidly as a discipline over the past two decades. An examination of how publication patterns and other scholarly activities of implementation scientists are weighted in the tenure and promotion process is needed given the unique and applied focus of the field.

Methods

We surveyed implementation scientists (mostly from the USA) to understand their perspectives on the following matters: (1) factors weighted in tenure and promotion for implementation scientists, (2) how important these factors are for success as an implementation scientist, (3) how impact is defined for implementation scientists, (4) top journals in implementation science, and (5) how these journals are perceived with regard to their prestige. We calculated univariate descriptive statistics for all quantitative data, and we used Wilcoxon signed-rank tests to compare the participants’ ratings of various factors. We analyzed open-ended qualitative responses using content analysis.

Results

One hundred thirty-two implementation scientists completed the survey (response rate = 28.9%). Four factors were rated as more important for tenure and promotion decisions: number of publications, quality of publication outlets, success in obtaining external funding, and record of excellence in teaching. Six factors were rated as more important for overall success as an implementation scientist: presentations at professional meetings, involvement in professional service, impact of the implementation scientist’s scholarship on the local community and/or state, impact of the implementation scientist’s scholarship on the research community, the number and quality of the implementation scientist’s community partnerships, and the implementation scientist’s ability to disseminate their work to non-research audiences. Participants most frequently defined and described impact as changing practice and/or policy. This expert cohort identified Implementation Science as the top journal in the field.

Conclusions

Overall, there was a significant mismatch between the factors experts identified as being important to academic success (e.g., tenure and promotion) and the factors needed to be a successful implementation scientist. Findings have important implications for capacity building, although they are largely reflective of the promotion and tenure process in the USA.

Supplementary Information

The online version contains supplementary material available at 10.1186/s43058-022-00323-0.

Federal Funding and Citation Metrics of US Biomedical Researchers, 1996 to 2022

IMPORTANCE

Both citation and funding metrics converge in shaping current perceptions of academic success.

OBJECTIVE

To evaluate what proportion of the most-cited US-based scientists are funded by biomedical federal agencies and whether funded scientists are more cited than nonfunded ones.

DESIGN, SETTING, AND PARTICIPANTS

This survey study used linkage of a Scopus-based database on top-cited US researchers (according to a composite citation metric) and the National Institutes of Health RePORTER database of federal funding (33 biomedical federal agencies). Matching was based on name and institution. US-based top-cited scientists who were allocated to any of 69 scientific subfields highly related to biomedicine were considered in the main analysis. Data were downloaded on June 11, 2022.

MAIN OUTCOMES AND MEASURES

Proportion of US-based top-cited biomedical scientists who had any (1996-2022), recent (2015-2022), and current (2021-2022) funding. Comparisons of funded and nonfunded scientists assessed total citations and a composite citation index.

RESULTS

There were 204 603 records in RePORTER (1996-2022) and 75 316 US-based top-cited scientists in the career-long citation database; 40 887 scientists were included in the main analysis. The proportion of US-based top-cited biomedical scientists (according to career-long citation impact) who had received any federal funding from biomedical research agencies was 62.7% (25 650 of 40 887) for any funding (1996-2022), 23.1% (9427 of 40 887) for recent funding (2015-2022), and 14.1% (5778 of 40 887) for current funding (2021-2022). Respective proportions were 64.8%, 31.4%, and 20.9%, for top-cited scientists according to recent single-year citation impact. There was large variability across scientific subfields (eg, current funding: 31% of career-long impact top-cited scientists in geriatrics, 30% in bioinformatics and 29% in developmental biology, but 0% in legal and forensic medicine, general psychology and cognitive sciences, and gender studies). Funded top-cited researchers were overall more cited than nonfunded top-cited scientists (median [IQR], 9594 [5650-1703] vs 5352 [3057-9890] citations; P < .001) and substantial difference remained after adjusting for subfield and years since first publication. Differences were more prominent in some specific biomedical subfields.

CONCLUSIONS AND RELEVANCE

In this survey study, biomedical federal funding had offered support to approximately two-thirds of the top-cited biomedical scientists at some point during the last quarter century, but only a small minority of top-cited scientists had current federal biomedical funding. The large unevenness across subfields needs to be addressed with ways that improve equity, efficiency, excellence, and translational potential.

Recognition of knowledge translation practice in Canadian health sciences tenure and promotion: A content analysis of institutional policy documents

BACKGROUND AND OBJECTIVE

There has been growing emphasis on increasing impacts of academic health research by integrating research findings in healthcare. The concept of knowledge translation (KT) has been widely adopted in Canada to guide this work, although lack of recognition in tenure and promotion (T&P) structures have been identified as barrier to researchers undertaking KT. Our objective was to explore how KT is considered in institutional T&P documentation in Canadian academic health sciences.

METHODS

We conducted content analysis of T&P documents acquired from 19 purposively sampled research-intensive or largest regional Canadian institutions in 2020-2021. We coded text for four components of KT (synthesis, dissemination, exchange, application). We identified clusters of related groups of documents interpreted together within the same institution. We summarized manifest KT content with descriptive statistics and identified latent categories related to how KT is considered in T&P documentation.

RESULTS

We acquired 89 unique documents from 17 institutions that formed 48 document clusters. Most of the 1057 text segments were categorized as dissemination (n = 851, 81%), which was included in 47 document clusters (98%). 15 document clusters (31%) included all four KT categories, while one (2%) did not have any KT categories identified. We identified two latent categories: primarily implicit recognition of KT; and an overall lack of clarity on KT.

CONCLUSIONS

Our analysis of T&P documents from primarily research-intensive Canadian universities showed a lack of formal recognition for a comprehensive approach to KT and emphasis on traditional dissemination. We recommend that institutions explicitly and comprehensively consider KT in T&P and align documentation and procedures to reflect these values.

Research features between Urology and Nephrology authors in articles regarding UTI related to CKD, HD, PD, and renal transplantation

BACKGROUND

A urinary tract infection (UTI) is one of the most common types of infections affecting the urinary tract. When bacteria enter the bladder or kidney and multiply in the urine, a URI can occur. The urethra is shorter in women than in men, which makes it easier for bacteria to reach the bladder or kidneys and cause infection. A comparison of the research differences between Urology and Nephrology (UN) authors regarding UTI pertaining to the 4 areas (i.e., Chronic Kidney Disease, Hemodialysis, Peritoneal Dialysis, and Renal Transplantation [CHPR]) is thus necessary. We propose and verify 2 hypotheses: CHPR-related articles on UTI have equal journal impact factors (JIFs) in research achievements (RAs) and UN authors have similar research features (RFs).

METHODS

Based on keywords associated with UTI and CHPR in titles, subject areas, and abstracts since 2013, we obtained 1284 abstracts and their associated metadata (e.g., citations, authors, research institutes, departments, countries of origin) from the Web of Science core collection. There were 1030 corresponding and first (co-first) authors with hT-JIF-indices (i.e., JIF was computed using hT-index rather than citations as usual). The following 5 visualizations were used to present the author's RA: radar, Sankey, time-to-event, impact beam plot, and choropleth map. The forest plot was used to distinguish RFs by observing the proportional counts of keyword plus in Web of Science core collection between UN authors.

RESULTS

It was observed that CHPR-related articles had unequal JIFs (χ2 = 13.08, P = .004, df = 3, n = 1030) and UN departments had different RFs (Q = 53.24, df = 29, P = .004). In terms of countries, institutes, departments, and authors, the United States (hT-JIF = 38.30), Mayo Clinic (12.9), Nephrology (19.14), and Diana Karpman (10.34) from Sweden had the highest hT-JIF index.

CONCLUSION

With the aid of visualizations, the hT-JIF-index and keyword plus were demonstrated to assess RAs and distinguish RFs between UN authors. A replication of this study under other topics and in other disciplines is recommended in the future, rather than limiting it to UN authors only, as we did in this study.

Does Research Training lead to Academic Success in Orthopedic Surgery? An Analysis of U.S Academic Orthopedic Surgeons

BACKGROUND

Academic surgeons are invaluable for scientific advancement and training the next generation of orthopedic surgeons.

OBJECTIVE

This study aimed to describe a cohort of academic orthopedic surgeons currently in practice with common academic metrics.

METHODS

ACGME-accredited orthopedic surgery programs with a university affiliation were identified. The primary independent variable in this study was formal research training as defined by a research fellowship or attainment of a PhD. Outcomes included academic rank, h-index attained, number of publications, and funding by the National Institutes of Health (NIH).

RESULTS

1641 orthopedic surgeons were identified across 73 programs. 116 surgeons (7.07%) received formal academic research training. The academic training group and non-academic training group had a similar completion rate of clinical fellowship programs (93.97% vs 93.77%, p=0.933), attainment of other advanced degrees (10.34% vs 8.46%, p=0.485), and years since completion of training (17.49-years vs 16.28-years, p=0.284). Surgeons completing academic research training had a significantly higher h-index (18.46 vs 10.88, p<0.001), higher publication number (67.98 vs 37.80, p<0.001), and more likely to be NIH funded (16.38% vs 3.15%, p<0.001). Surgeons completing academic training were more likely to be associate professors (34.48% vs 25.77%), professors (25.00% vs 22.82%), and endowed professors (10.34% vs 2.43%) (p<0.001). On regression analysis, formalized research training was independently associated with h-index and NIH funding (p<0.001 for both).

CONCLUSION

Formalized research training, either as a research fellowship or PhD, is associated with an increased h-index and likelihood of NIH funding, although this association was not found for academic rank after adjusted regression analysis.

Gender Differences in National Institutes of Health Grant Submissions Before and During the COVID-19 Pandemic

Introduction: Emerging data suggest that the COVID-19 pandemic has disproportionately impacted women in academic medicine, potentially eliminating recent gains that have been made toward gender equity. This study examined possible pandemic-related gender disparities in research grant submissions, one of the most important criteria for academic promotion and tenure evaluations. Methods: Data were collected from two major academic institutions (one private and one public) on the gender and academic rank of faculty principal investigators who submitted new grants to the National Institutes of Health (NIH) during COVID-19 (March 1st, 2020, through August 31, 2020) compared with a matched period in 2019 (March 1st, 2019, through August 31, 2019). t-Tests and chi-square analyses compared the gender distribution of individuals who submitted grants during the two periods of examination. Results: In 2019 (prepandemic), there was no significant difference in the average number of grants submitted by women compared with men faculty. In contrast, women faculty submitted significantly fewer grants in 2020 (during the pandemic) than men. Men were also significantly more likely than women to submit grants in both 2019 and 2020 compared with submitting in 2019 only, suggesting men faculty may have been more likely than their women colleagues to sustain their productivity in grant submissions during the pandemic. Discussion: Women's loss of extramural funding may compound over time, as it impedes new data collection, research progress, and academic advancement. Efforts to support women's research productivity and career trajectories are urgently needed in the following years of pandemic recovery.

Lessons from the COVID-19 pandemic and recent developments on the communication of clinical trials, publishing practices, and research integrity: in conversation with Dr. David Moher

BACKGROUND

The torrent of research during the coronavirus (COVID-19) pandemic has exposed the persistent challenges with reporting trials, open science practices, and scholarship in academia. These real-world examples provide unique learning opportunities for research methodologists and clinical epidemiologists-in-training. Dr. David Moher, a recognized expert on the science of research reporting and one of the founders of the Consolidated Standards of Reporting Trials (CONSORT) statement, was a guest speaker for the 2021 Hooker Distinguished Visiting Professor Lecture series at McMaster University and shared his insights about these issues.

MAIN TEXT

This paper covers a discussion on the influence of reporting guidelines on trials and issues with the use of CONSORT as a measure of quality. Dr. Moher also addresses how the overwhelming body of COVID-19 research reflects the "publish or perish" paradigm in academia and why improvement in the reporting of trials requires policy initiatives from research institutions and funding agencies. We also discuss the rise of publication bias and other questionable reporting practices. To combat this, Dr. Moher believes open science and training initiatives led by institutions can foster research integrity, including the trustworthiness of researchers, institutions, and journals, as well as counter threats posed by predatory journals. He highlights how metrics like journal impact factor and quantity of publications also harm research integrity. Dr. Moher also discussed the importance of meta-science, the study of how research is carried out, which can help to evaluate audit and feedback systems and their effect on open science practices.

CONCLUSION

Dr. Moher advocates for policy to further improve the reporting of trials and health research. The COVID-19 pandemic has exposed how a lack of open science practices and flawed systems incentivizing researchers to publish can harm research integrity. There is a need for a culture shift in assessing careers and "productivity" in academia, and this requires collaborative top-down and bottom-up approaches.

Questionable Research Practices, Low Statistical Power, and Other Obstacles to Replicability: Why Preclinical Neuroscience Research Would Benefit from Registered Reports

Replicability, the degree to which a previous scientific finding can be repeated in a distinct set of data, has been considered an integral component of institutionalized scientific practice since its inception several hundred years ago. In the past decade, large-scale replication studies have demonstrated that replicability is far from favorable, across multiple scientific fields. Here, I evaluate this literature and describe contributing factors including the prevalence of questionable research practices (QRPs), misunderstanding of p-values, and low statistical power. I subsequently discuss how these issues manifest specifically in preclinical neuroscience research. I conclude that these problems are multifaceted and difficult to solve, relying on the actions of early and late career researchers, funding sources, academic publishers, and others. I assert that any viable solution to the problem of substandard replicability must include changing academic incentives, with adoption of registered reports being the most immediately impactful and pragmatic strategy. For animal research in particular, comprehensive reporting guidelines that document potential sources of sensitivity for experimental outcomes is an essential addition.

Altmetric score, reads, and citations in paediatric cardiology: do they correlate and what do they mean?

INTRODUCTION

The primary objective of this study was to determine whether Altmetric score, number of reads, and citations for paediatric cardiology manuscripts correlate with one another. A secondary objective was to determine the extent to which factors mediated citation number for paediatric cardiology manuscripts.

METHODS

Data for this study came from manuscripts published in Cardiology in the Young (2010-2021). Data were extracted by using data shared on the journal website. Spearman's correlation analyses were conducted between manuscript reads, citations, and Altmetric score. Regression analyses were conducted with number of citations as the dependent variable and year of publication, publication type, number of reads, and Altmetric score as independent variables.

RESULTS

A total of 2642 manuscripts were included in the final analyses. Reads and citations had poor correlation (r-value 0.32); reads and Altmetric score had negligible correlation (r-value 0.26); and Altmetric score and citations had negligible correlation (r-value 0.07). Year of publication was independently associated with number of citations (β -0.95, p-value <0.01). Manuscript type was independently associated with number of citations (β 1.04, p-value <0.01). Number of reads was independently associated with citations (β 0.01, p-value <0.01). Altmetric score was independently associated with number of citations (β 0.05, p-value <0.01).

CONCLUSION

This study describes the correlation of reads, citations, and Altmetric score in manuscripts published in Cardiology in the Young, demonstrating poor correlation, at best, between these metrics. Each bibliometric index seems to represent a different phenomenon of manuscript consumption. No single bibliometric index in isolation offers ample representation of manuscript consumption.

Data Sharing and Reanalyses Among Randomized Clinical Trials Published in Surgical Journals Before and After Adoption of a Data Availability and Reproducibility Policy

Importance

Clinical trial data sharing holds promise for maximizing the value of clinical research. The International Committee of Medical Journal Editors (ICMJE) adopted a policy promoting data sharing in July 2018.

Objective

To evaluate the association of the ICMJE data sharing policy with data availability and reproducibility of main conclusions among leading surgical journals.

Design, Setting, and Participants

This cross-sectional study, conducted in October 2021, examined randomized clinical trials (RCTs) in 10 leading surgical journals before and after the implementation of the ICMJE data sharing policy in July 2018.

Exposure

Implementation of the ICMJE data sharing policy.

Main Outcomes and Measures

To demonstrate a pre-post increase in data availability from 5% to 25% (α = .05; β = 0.1), 65 RCTs published before and 65 RCTs published after the policy was issued were included, and their data were requested. The primary outcome was data availability (ie, the receipt of sufficient data to enable reanalysis of the primary outcome). When data sharing was available, the primary outcomes reported in the journal articles were reanalyzed to explore reproducibility. The reproducibility features of these studies were detailed.

Results

Data were available for 2 of 65 RCTs (3.1%) published before the ICMJE policy and for 2 of 65 RCTs (3.1%) published after the policy was issued (odds ratio, 1.00; 95% CI, 0.07-14.19; P > .99). A data sharing statement was observed in 11 of 65 RCTs (16.9%) published after the policy vs none before the policy (risk ratio, 2.20; 95% CI, 1.81-2.68; P = .001). Data obtained for reanalysis (n = 4) were not from RCTs published with a data sharing statement. Of the 4 RCTs with available data, all of them had primary outcomes that were fully reproduced. However, discrepancies or inaccuracies that were not associated with study conclusions were identified in 3 RCTs. These concerned the number of patients included in 1 RCT, the management of missing values in another RCT, and discrepant timing for the principal outcome declared in the study registration and reported in the third RCT.

Conclusions and Relevance

This cross-sectional study suggests that data sharing practices are rare in surgical journals despite the ICMJE policy and that most RCTs published in these journals lack transparency. The results of these studies may not be reproducible by external researchers.

Improving medical research in the United Kingdom

Poor quality medical research causes serious harms by misleading healthcare professionals and policymakers, decreasing trust in science and medicine, and wasting public funds. Here we outline underlying problems including insufficient transparency, dysfunctional incentives, and reporting biases. We make the following recommendations to address these problems: Journals and funders should ensure authors fulfil their obligation to share detailed study protocols, analytical code, and (as far as possible) research data. Funders and journals should incentivise uptake of registered reports and establish funding pathways which integrate evaluation of funding proposals with initial peer review of registered reports. A mandatory national register of interests for all those who are involved in medical research in the UK should be established, with an expectation that individuals maintain the accuracy of their declarations and regularly update them. Funders and institutions should stop using metrics such as citations and journal's impact factor to assess research and researchers and instead evaluate based on quality, reproducibility, and societal value. Employers and non-academic training programmes for health professionals (clinicians hired for patient care, not to do research) should not select based on number of research publications. Promotions based on publication should be restricted to those hired to do research.

Faculty appointment and promotion in Taiwan's medical schools, a systematic analysis

BACKGROUND

A rigorous faculty appointment and promotion (FAP) system is vital for the success of any academic institution. However, studies examining the FAP system in Asian universities are lacking. We surveyed the FAP policies of Taiwan's medical schools and identified an overreliance on the CJA score (manuscript Category, Journal quality, and Author order). The potential shortcomings of this metric and recommendations for refinement were discussed.

METHODS

We obtained the FAP documents from all 12 medical schools in Taiwan, and analyzed their use of traditional versus non-traditional criteria for FAP according to a published methodology. The influence of the journal impact factor (JIF) on the FAP process was quantified by comparing its relative weight between papers with two extreme JIFs. To better understand the research impact and international standing of each school, we utilized the public bibliographic database to rank universities by the number of papers, and the proportions of papers within the top 10% or 50% citation.

RESULTS

Compared with other countries, Taiwan's medical schools focus more on the quantifiable quality of the research, mostly using a "CJA" score that integrates the category, JIF or ranking, and authorship of a paper, with the JIF being the most influential factor. The CJA score for an article with a JIF of 20 can be up to three times the threshold for promotion to Assistant Professor. The emphasis on JIF is based on a presumed correlation between JIF and citation counts. However, our analysis shows that Taiwan's medical schools have lower-than-average citation counts despite a competitive rank in the number of publications.

CONCLUSIONS

The JIF plays an unrivaled role in determining the outcome of FAP in Taiwan's medical schools, mostly via the CJA system. The questionable effectiveness of the current system in elevating the international standing of Taiwan's higher-education institutions calls for a re-examination of the FAP system. We recommend a reduction in the relative importance of CJA score in the FAP system, adopting more rigorous metrics such as the h-index for evaluating research quality, and supporting more research aimed at improving the FAP system.

Do German university medical centres promote robust and transparent research? A cross-sectional study of institutional policies

Background

In light of replication and translational failures, biomedical research practices have recently come under scrutiny. Experts have pointed out that the current incentive structures at research institutions do not sufficiently incentivise researchers to invest in robustness and transparency and instead incentivise them to optimize their fitness in the struggle for publications and grants. This cross-sectional study aimed to describe whether and how relevant policies of university medical centres in Germany support the robust and transparent conduct of research and how prevalent traditional metrics are.

Methods

For 38 German university medical centres, we searched for institutional policies for academic degrees and academic appointments as well as websites for their core facilities and research in general between December 2020 and February 2021. We screened the documents for mentions of indicators of robust and transparent research (study registration; reporting of results; sharing of research data, code and protocols; open access; and measures to increase robustness) and for mentions of more traditional metrics of career progression (number of publications; number and value of awarded grants; impact factors; and authorship order).

Results

While open access was mentioned in 16% of PhD regulations, other indicators of robust and transparent research were mentioned in less than 10% of institutional policies for academic degrees and academic appointments. These indicators were more frequently mentioned on the core facility and general research websites. Institutional policies for academic degrees and academic appointments had frequent mentions of traditional metrics.

Conclusions

References to robust and transparent research practices are, with a few exceptions, generally uncommon in institutional policies at German university medical centres, while traditional criteria for academic promotion and tenure still prevail.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12961-022-00841-2.

Grant writing and grant peer review as questionable research practices

A large part of governmental research funding is currently distributed through the peer review of project proposals. In this paper, we argue that such funding systems incentivize and even force researchers to violate five moral values, each of which is central to commonly used scientific codes of conduct. Our argument complements existing epistemic arguments against peer-review project funding systems and, accordingly, strengthens the mounting calls for reform of these systems.

Meta-research: Justifying career disruption in funding applications, a survey of Australian researchers

Background:

When researchers’ careers are disrupted by life events – such as illness or childbirth – they often need to take extended time off. This creates a gap in their research output that can reduce their chances of winning funding. In Australia, applicants can disclose their career disruptions and peer reviewers are instructed to make appropriate adjustments. However, it is not clear if and how applicants use career disruption sections or how reviewers adjust and if they do it consistently.

Methods:

To examine career disruption, we used surveys of the Australian health and medical research community. We used both a random sample of Australian authors on PubMed and a non-random convenience sample.

Results:

Respondents expressed concerns that sharing information on career disruption would harm their chances of being funded, with 13% saying they have medical or social circumstances but would not include it in their application, with concerns about appearing ‘weak’. Women were more reluctant to include disruption. There was inconsistency in how disruption was adjusted for, with less time given for those with depression compared with caring responsibilities, and less time given for those who did not provide medical details of their disruption.

Conclusions:

The current system is likely not adequately adjusting for career disruption and this may help explain the ongoing funding gap for senior women in Australia.

Funding:

National Health and Medical Research Council Senior Research Fellowship (Barnett).

Science is an expensive endeavor. To pursue their ideas, most researchers need to win funding by submitting applications to highly competitive schemes with low success rates. Funding decisions depend on many factors, but usually take into consideration a researcher’s track record: publications, collaborations with other researchers and even other awards they have received.

Researchers whose careers have been disrupted by life events, including childbearing or being ill, may have a gap in their track record that reduces their chances of winning funding. Historically, female researchers have experienced career disruptions more often, leading to a funding gap between male and female researchers. To increase fairness and reduce this gap, many funding agencies have instructed the peer reviewers – other scientists – who assess funding applications to adjust their scores to account for career disruptions. However, large funding gaps are still frequently observed between female and male researchers.

Barnett et al. wanted to know how career disruption is considered in practice by establishing what personal details are shared in applications by researchers with disruption, and how reviewers treat this information. To find out, they surveyed medical researchers in Australia and asked them for their views on career disruption as both funding applicants and reviewers of funding applications.

The answers to the survey indicated that 13% of the applicants responding had experienced career disruptions, but would not include them in funding applications. In many cases, this reluctance to disclose career disruptions was due to concerns that it would harm an applicant’s chances of winning funding, a concern that was greater in the women who responded to the survey. Researchers who answered the survey would claim less time off on average if their career disruption was for severe depression compared with caring for a child or elderly relative. Additionally, the answers to the survey show that, on average, peer reviewers – the scientists who assessed the applications – would give more time off to applicants who provided details about the medical issues that caused a career disruption than to those who did not.

The results of this survey suggest that changes in the systems used to apply for funding and in how applications are assessed could make funding fairer. One suggestion would be to modify funding applications to make disruptions easier to report. Another would be to make changes to the reviewing procedures to increase privacy and reduce variability in how disruption is assessed. Changes in these directions could help researchers gain access to funding more fairly, increasing the quality and output of scientific research.

What senior academics can do to support reproducible and open research: a short, three-step guide

Increasingly, policies are being introduced to reward and recognise open research practices, while the adoption of such practices into research routines is being facilitated by many grassroots initiatives. However, despite this widespread endorsement and support, as well as various efforts led by early career researchers, open research is yet to be widely adopted. For open research to become the norm, initiatives should engage academics from all career stages, particularly senior academics (namely senior lecturers, readers, professors) given their routine involvement in determining the quality of research. Senior academics, however, face unique challenges in implementing policy changes and supporting grassroots initiatives. Given that—like all researchers—senior academics are motivated by self-interest, this paper lays out three feasible steps that senior academics can take to improve the quality and productivity of their research, that also serve to engender open research. These steps include changing (a) hiring criteria, (b) how scholarly outputs are credited, and (c) how we fund and publish in line with open research principles. The guidance we provide is accompanied by material for further reading.