The Matthew effect in science funding

Funding priorities and health outcomes in Danish medical research

External research funding is an essential component of the infrastructure of modern, academic research. Priorities in funding decisions drive what knowledge is generated, and how scientists' careers are shaped. For health research, it can ultimately have implications for health outcomes. The aim of this paper is to illustrate how funding information can be used to track priorities in health research, linking them to disease burdens and research outputs. Furthermore, funding concentrations are analysed from both researcher and disease perspectives, to estimate the influence of personal Matthew-effects on the distribution of health research funding. Denmark is used as the case, including funding information from all major public and private research foundations in the period 2004-2016. Grant information is linked to research outputs and disability-adjusted life-years (DALY rates), for 34,160 publications linked to 2630 grants, receiving DKK 4.8 billion in funding. Data show poor correlation between funding priorities, research activity and disease burdens, with several diseases receiving disproportionate amounts of funding. A research opportunity index is calculated to identify diseases with the highest potential for future investments from a burden-centred point of view. Funding is highly concentrated, both on people and on specific diseases. High funding concentrations on researchers can be a driving factor behind the observed funding-to-burden imbalances, and may risk knowledge stagnation through monopolisation of the market place of ideas. Results indicate that funders of clinical and translational research, as well as some types of biomedical research, need to supplement traditional considerations of scientific excellence with measures of societal challenges and relevance.

"Best Paper" awards lack transparency, inclusivity, and support for Open Science

Awards can propel academic careers. They also reflect the culture and values of the scientific community. But do awards incentivize greater transparency, inclusivity, and openness in science? Our cross-disciplinary survey of 222 awards for the "best" journal articles across all 27 SCImago subject areas revealed that journals and learned societies administering such awards generally publish little detail on their procedures and criteria. Award descriptions were brief, rarely including contact details or information on the nominations pool. Nominations of underrepresented groups were not explicitly encouraged, and concepts that align with Open Science were almost absent from the assessment criteria. At the same time, 10% of awards, especially the recently established ones, tended to use article-level impact metrics. USA-affiliated researchers dominated the winner's pool (48%), while researchers from the Global South were uncommon (11%). Sixty-one percent of individual winners were men. Overall, Best Paper awards miss the global calls for greater transparency and equitable access to academic recognition. We provide concrete and implementable recommendations for scientific awards to improve the scientific recognition system and incentives for better scientific practice.

Promotional language and the adoption of innovative ideas in science

How are the merits of innovative ideas communicated in science? Here, we conduct semantic analyses of grant application success with a focus on scientific promotional language, which may help to convey an innovative idea's originality and significance. Our analysis attempts to surmount the limitations of prior grant studies by examining the full text of tens of thousands of both funded and unfunded grants from three leading public and private funding agencies: the NIH, the NSF, and the Novo Nordisk Foundation, one of the world's largest private science funding foundations. We find a robust association between promotional language and the support and adoption of innovative ideas by funders and other scientists. First, a grant proposal's percentage of promotional language is associated with up to a doubling of the grant's probability of being funded. Second, a grant's promotional language reflects its intrinsic innovativeness. Third, the percentage of promotional language is predictive of the expected citation and productivity impact of publications that are supported by funded grants. Finally, a computer-assisted experiment that manipulates the promotional language in our data demonstrates how promotional language can communicate the merit of ideas through cognitive activation. With the incidence of promotional language in science steeply rising, and the pivotal role of grants in converting promising and aspirational ideas into solutions, our analysis provides empirical evidence that promotional language is associated with effectively communicating the merits of innovative scientific ideas.

Where next for partial randomisation of research funding? The feasibility of RCTs and alternatives

We outline essential considerations for any study of partial randomisation of research funding, and consider scenarios in which randomised controlled trials (RCTs) would be feasible and appropriate. We highlight the interdependence of target outcomes, sample availability and statistical power for determining the cost and feasibility of a trial. For many choices of target outcome, RCTs may be less practical and more expensive than they at first appear (in large part due to issues pertaining to sample size and statistical power). As such, we briefly discuss alternatives to RCTs. It is worth noting that many of the considerations relevant to experiments on partial randomisation may also apply to other potential experiments on funding processes (as described in The Experimental Research Funder's Handbook. RoRI, June 2022).

Association Between Institution ZIP Code Characteristics and NIH Funding

Introduction. While racial NIH funding disparities have been identified, little is known about the link between community demographics of institutions and NIH funding. We sought to evaluate the association between institution zip code characteristics and NIH funding. Methods. We linked the 2011-2021 NIH RePORTER database to Census data. We calculated the funding to each institution and stratified institutions into funding quartiles. We defined out independent variables as institution ZIP code level race/ethnicity (White, Black, and Hispanic), and socioeconomic status (household income, high school graduation rate, and unemployment rate). We used ordinal regression models to evaluate the association between institution ZIP code characteristics and grant funding quartile. Results. We included 731,548 grants (US$271,495,839,744) from 3,971 ZIP codes. The funding amounts in millions of U.S. dollars for the funding quartiles were fourth - 0.25, third - 1.1, second - 3.8, first - 43.5. Using ordinal regression, we found an association between increasing unemployment rate (OR = 1.03 [1.02, 1.05]), increasing high school graduation rate (OR = 3.6 [1.6, 8.4]), decreasing proportion of White people (OR = 0.4 [0.3, 0.5]), increasing proportion of Black people (OR = 1.3 [0.9, 1.8]), and increasing proportion of Hispanic/Latine people (OR = 2.5 [1.7, 3.5]) and higher grant funding quartiles. We found no association between household income and grant funding quartile. Conclusion. We found ZIP code demographics to be inadequate for evaluating NIH funding disparities, and the association between institution ZIP code demographics and investigator demographics is unclear. To evaluate and improve grant funding disparities, better grant recipient data accessibility and transparency are needed.

Not so fast with fast funding

In the wake of the COVID-19 pandemic, many have become increasingly dissatisfied with how science funding is distributed. Traditional grant funding processes are seen as stifling the creativity of researchers, in addition to being bureaucratic, slow, and inefficient. Consequently, there have been increasing popular calls to make "fast funding" - fast, unbureaucratic grant applications - a new standard for scientific funding. Though this approach to funding, implemented by Fast Grants, has been successful as a pandemic response strategy, we believe there are serious costs to its wide-scale adoption, particularly for transparency and equity, and that the purported benefits - increased creativity and efficiency - are unlikely to materialize. While traditional funding mechanisms are certainly not perfect, scientific communities should think twice before adopting fast funding as a new standard for funding.

The landscape of biomedical research funding in Brazil: a current overview

OBJECTIVE

The objective of this narrative review is to discuss the current state of research funding in Brazil.

MATERIALS AND METHODS

This study is based on the most recent edition of the course Funding for Research and Innovation in the University of Sao Paulo School of Medicine which was a three-day course with 12 hours of instruction. The course brought together leading experts in the field to comprehensively discuss the current state of research funding in Brazil. Each speaker provided a presentation on a specific topic related to research funding. After the workshop, speakers assembled relevant topics in this manuscript.

RESULTS

collaborative research is critical for securing research funding. It optimizes proposal competitiveness, amplifies societal impact, and manages risks effectively. As such, fostering and supporting these collaborations is paramount for both researchers and funding agencies. To maintain the highest integrity in research, investigators involved in these collaborations must disclose any relationships that could potentially influence the outcomes or interpretation of their projects.

CONCLUSIONS

In Brazil, the mainstay of research funding stems from public entities, with agencies such as CNPq, CAPES, and state bodies like FAPESP, FAPERJ, FAPEMIG and others at the forefront. Concurrently, industry funding offers viable pathways, especially through industry-sponsored studies, investigator-led projects, and collaborative initiatives. The Brazilian funding landscape is further enriched by innovative platforms, including crowdfunding and the contributions of institutions like the Serrapilheira Institute. Internationally, esteemed organizations such as the National Institutes of Health (NIH) and the Bill & Melinda Gates Foundation stand out as potential funders.

Comprehensive landscape and future perspectives of non-coding RNAs in esophageal squamous cell carcinoma, a bibliometric analysis from 2008 to 2023

Objectives: Summarize the progress and hot topic evolution of non-coding RNAs (ncRNAs) research in esophageal squamous cell carcinoma (ESCC) in recent years and predict future research directions. Methods: Relevant articles from the Web of Science until 31 October 2023 were obtained. Bibliometric analysis of included articles was performed using software (VOSviewer, CiteSpace, and Bibliometrix). The volume and citation of publications, as well as the country, institution, author, journal, keywords of the articles were used as variables to analyze the research trends and hot spot evolution. Results: 1,118 literature from 2008 to 2023 were retrieved from database, with 25 countries/regions, 793 institutions, 5,426 authors, 261 journals involved. Global cooperation was centered on China, Japan, and the United States. Zhengzhou University, an institution from China, had the highest publication. The most prolific author was Guo Wei, and the most prolific journal was Oncology Letters. Analysis of keywords revealed that the research in this field revolved around the role of ncRNAs in the occurrence, development, diagnosis, treatment, and prognosis of ESCC, mainly including micro RNAs, long non-coding RNAs, and then circular RNAs. Conclusion: Overall, research on ncRNAs in ESCC remains strong. Previous research has mainly focused on the basic research, with a focus on the mechanism of ncRNAs in the occurrence, development, diagnosis, treatment, and prognosis of ESCC. Combining current research with emerging disciplines to further explore its mechanisms of action or shifting the focus of research from preclinical research to clinical research based on diagnosis, treatment, and prognosis, will be the main breakthrough in this field in the future.

Gender differences in Dutch research funding over time: A statistical investigation of the innovation scheme 2012-2021

BACKGROUND

In 2015, the Dutch research council, NWO, took measures to combat gender bias disadvantaging female applicants in a popular three-tiered funding scheme called the Talent Programme. The innovation scheme consists of three grants for different career stages, called Veni, Vidi and Vici.

OBJECTIVES

This paper studies the question whether or not NWO has been successful in removing gender differences in their funding procedure.

METHODS

Using all available data from 2012 onwards of grant applications in the Talent Programme (16,249 applications of which 2,449 received funding), we study whether these measures had an effect using binomial generalized linear models.

RESULTS

We find strong statistical evidence of a shift in gender effects in favour of female applicants in the first tier, the Veni (p < .001). Significant gender differences are not found in the two other tiers, the Vidi and Vici schemes.

CONCLUSIONS

In recent years, female applicants are more likely to be awarded with a Veni grant than male applicants and this gender gap has increased over time. This suggests that gender differences still exist in the assessment of Talent Programme submissions, albeit in a different direction than a decade ago.

Structural Power and Epistemologies in the Scientific Field: Why a Rapid Reconciliation Between Functional and Evolutionary Biology is Unlikely

The past decade has been marked by a series of global crises, presenting an opportunity to reevaluate the relationship between science and politics. The biological sciences are instrumental in understanding natural phenomena and informing policy decisions. However, scholars argue that current scientific expertise often fails to account for entire populations and long-term impacts, hindering efforts to address issues such as biodiversity loss, global warming, and pandemics. This article explores the structural challenges of integrating an evolutionary perspective, historically opposed to functional determinants of health and disease, into current biological science practices. Using data on Swiss biology professors from 1957, 1980, and 2000, we examine the structural power dynamics that have led to the division between these competing epistemologies, and how this division has influenced resource allocation and career trajectories. Our analysis suggests that this cleavage presents a significant obstacle to achieving fruitful reconciliations, and that increased academicization and internationalization may benefit functional biologists at the expense of evolutionary biologists. While evolutionary biologists have gained symbolic recognition in recent years, this has not translated into valuable expertise in the political domain.

Trends in research grant applications and outcomes among medical students in the United Kingdom: a national self-reported cross-sectional survey

BACKGROUND

Research funding disparities contribute to clinical academic workforce inequalities. Hence, our study explores the association between student demographics and research grant application rates and outcomes among UK medical students.

METHODS

This is a national multicentre cross-sectional survey of UK medical students in the 2020-21 academic year. Multiple zero-inflated negative binomial regression and generalized linear model (binomial distribution; logit link) were utilized to investigate the association between student demographics, number of grant applications submitted, and successful grant applications (yes or no). P-values less than a Bonferroni-corrected significance level of 0.05/36 = 0.0014 were considered to be statistically significant.

RESULTS

A total of 1528 students participated from 36 medical schools. One hundred fifty-one respondents (9.9%) had applied for research grants. Black students submitted applications 2.90 times more often than white students [Incident rate ratio (IRR): 2.90, 95% confidence interval (CI): 1.37-6.16], with no ethnic disparity in the odds of successful applications. Gender did not influence application rates significantly (P = .248), but women were 4.61 times more likely to secure a grant than men [odds ratio: 4.61, 95% CI: 2.04-10.4]. Being a PubMed-indexed author was associated with increased grant application submission rates [IRR: 3.61, 95% CI: 2.20- 5.92] while conducting more research was associated with greater odds of securing a grant [odds ratio: 1.42, 95% CI: 1.17- 1.73].

CONCLUSION

Although black students submitted more applications, ethnicity did not influence success rates. Gender did not influence application rates, but women were more successful. These findings underscore the need for strategies supporting women and underrepresented students for continued academic achievement after graduation.

KEY MESSAGES

What is already known on this topic Research funding for post-PhD researchers is believed to be a major driver of gender and ethnic inequalities in the clinical academic workforce.Students who receive research grants are more likely to receive postgraduate research grants.What this study adds Black students applied for more research grants than white students, but there were no ethnic differences in the odds of securing a grant.There were no gender differences in the research grant application rates. However, female students had greater odds of securing research grants compared to male students.How this study might affect research, practice or policy Medical schools should incorporate grant writing skills into the undergraduate research curriculum. Also, to sustain women's academic success post medical school, the NIHR and affiliates should provide research award extensions and childcare support for women when required.

Analysis of NIH K99/R00 awards and the career progression of awardees

Many postdoctoral fellows and scholars who hope to secure tenure-track faculty positions in the United States apply to the National Institutes of Health (NIH) for a Pathway to Independence Award. This award has two phases (K99 and R00) and provides funding for up to 5 years. Using NIH data for the period 2006-2022, we report that ~230 K99 awards were made every year, representing up to ~$250 million annual investment. About 40% of K99 awardees were women and ~89% of K99 awardees went on to receive an R00 award annually. Institutions with the most NIH funding produced the most recipients of K99 awards and recruited the most recipients of R00 awards. The time between a researcher starting an R00 award and receiving a major NIH award (such as an R01) ranged between 4.6 and 7.4 years, and was significantly longer for women, for those who remained at their home institution, and for those hired by an institution that was not one of the 25 institutions with the most NIH funding. Shockingly, there has yet to be a K99 awardee at a historically Black college or university. We go on to show how K99 awardees flow to faculty positions, and to identify various factors that influence the future success of individual researchers and, therefore, also influence the composition of biomedical faculty at universities in the United States.

Scientific production in biodiversity: the gender gap continues in Brazilian universities

Scientometrics enables us to comprehend the interests and trends in scientific knowledge production and dissemination. In this study, we evaluate the effects of gender, academic experience, location of residence, and graduate program score on the quantity of published articles, the number of citations and the H-index of researchers belonging to Brazilian graduate programs in Biodiversity. Variables related to the researchers were measured, and the relevance in explaining scientific production was examined using hierarchical models. In graduate programs, there were more men than females. The number of articles as first author and the H-index increase progressively through the researchers' career, while the number of citations increases at the beginning of their careers, stabilizing between 10 and 20 years, and increasing again after 30 years of career. We concluded that gender, academic experience, and graduate program score were the most important variables in explaining the scientific production of graduate programs in Biodiversity in Brazil.

FENS-Kavli Network of Excellence: Postponed, non-competitive peer review for research funding

Receiving research grants is among the highlights of an academic career, affirming previous accomplishments and enabling new research endeavours. Much of the process of acquiring research funding, however, belongs to the less favourite duties of many researchers: It is time consuming, often stressful and, in the majority of cases, unsuccessful. This resentment towards funding acquisition is backed up by empirical research: The current system to distribute research funding, via competitive calls for extensive research applications that undergo peer review, has repeatedly been shown to fail in its task to reliably rank proposals according to their merit, while at the same time being highly inefficient. The simplest, fairest and broadly supported alternative would be to distribute funding more equally across researchers, for example, by an increase of universities' base funding, thereby saving considerable time that can be spent on research instead. Here, I propose how to combine such a 'funding flat rate' model-or other efficient distribution strategies-with quality control through postponed, non-competitive peer review using open science practices.

Gender inequality in cum laude distinctions for PhD students

Resource allocation in academia is highly skewed, and peer evaluation is the main method used to distribute scarce resources. A large literature documents gender inequality in evaluation, and the explanation for this inequality is homophily: male evaluators give more favorable ratings to male candidates. We investigate this by focusing on cum laude distinctions for PhD students in the Netherlands, a distinction that is only awarded to 5 percent of all dissertations and has as its sole goal to distinguish the top from the rest. Using data from over 5000 PhD recipients of a large Dutch university for the period 2011-2021, we find that female PhD students were almost two times less likely to get a cum laude distinction than their male counterparts, even when they had the same doctoral advisor. This gender gap is largest when dissertations are evaluated by all-male committees and decreases as evaluation committees include more female members.

Reviewer training for improving grant and journal peer review

BACKGROUND

Funders and scientific journals use peer review to decide which projects to fund or articles to publish. Reviewer training is an intervention to improve the quality of peer review. However, studies on the effects of such training yield inconsistent results, and there are no up-to-date systematic reviews addressing this question.

OBJECTIVES

To evaluate the effect of peer reviewer training on the quality of grant and journal peer review.

SEARCH METHODS

We used standard, extensive Cochrane search methods. The latest search date was 27 April 2022.

SELECTION CRITERIA

We included randomized controlled trials (RCTs; including cluster-RCTs) that evaluated peer review with training interventions versus usual processes, no training interventions, or other interventions to improve the quality of peer review.

DATA COLLECTION AND ANALYSIS

We used standard Cochrane methods. Our primary outcomes were 1. completeness of reporting and 2. peer review detection of errors. Our secondary outcomes were 1. bibliometric scores, 2. stakeholders' assessment of peer review quality, 3. inter-reviewer agreement, 4. process-centred outcomes, 5. peer reviewer satisfaction, and 6. completion rate and speed of funded projects. We used the first version of the Cochrane risk of bias tool to assess the risk of bias, and we used GRADE to assess the certainty of evidence.

MAIN RESULTS

We included 10 RCTs with a total of 1213 units of analysis. The unit of analysis was the individual reviewer in seven studies (722 reviewers in total), and the reviewed manuscript in three studies (491 manuscripts in total). In eight RCTs, participants were journal peer reviewers. In two studies, the participants were grant peer reviewers. The training interventions can be broadly divided into dialogue-based interventions (interactive workshop, face-to-face training, mentoring) and one-way communication (written information, video course, checklist, written feedback). Most studies were small. We found moderate-certainty evidence that emails reminding peer reviewers to check items of reporting checklists, compared with standard journal practice, have little or no effect on the completeness of reporting, measured as the proportion of items (from 0.00 to 1.00) that were adequately reported (mean difference (MD) 0.02, 95% confidence interval (CI) -0.02 to 0.06; 2 RCTs, 421 manuscripts). There was low-certainty evidence that reviewer training, compared with standard journal practice, slightly improves peer reviewer ability to detect errors (MD 0.55, 95% CI 0.20 to 0.90; 1 RCT, 418 reviewers). We found low-certainty evidence that reviewer training, compared with standard journal practice, has little or no effect on stakeholders' assessment of review quality in journal peer review (standardized mean difference (SMD) 0.13 standard deviations (SDs), 95% CI -0.07 to 0.33; 1 RCT, 418 reviewers), or change in stakeholders' assessment of review quality in journal peer review (SMD -0.15 SDs, 95% CI -0.39 to 0.10; 5 RCTs, 258 reviewers). We found very low-certainty evidence that a video course, compared with no video course, has little or no effect on inter-reviewer agreement in grant peer review (MD 0.14 points, 95% CI -0.07 to 0.35; 1 RCT, 75 reviewers). There was low-certainty evidence that structured individual feedback on scoring, compared with general information on scoring, has little or no effect on the change in inter-reviewer agreement in grant peer review (MD 0.18 points, 95% CI -0.14 to 0.50; 1 RCT, 41 reviewers, low-certainty evidence).

AUTHORS' CONCLUSIONS

Evidence from 10 RCTs suggests that training peer reviewers may lead to little or no improvement in the quality of peer review. There is a need for studies with more participants and a broader spectrum of valid and reliable outcome measures. Studies evaluating stakeholders' assessments of the quality of peer review should ensure that these instruments have sufficient levels of validity and reliability.

A normative theory of luck

Psychologists have identified heuristics and biases that can cause people to make assumptions about factors that contribute to the success of individuals and firms, whose outcomes may have actually resulted primarily from randomness. Yet the interpretation of these biases becomes ambiguous when they represent reasonable cognitive shortcuts that offer certain advantages. This paper addresses this ambiguity by presenting four versions (weak, semi-weak, semi-strong, strong) of a normative theory of luck that integrates insights from psychology with the chance model approach to predict the circumstances under which performance non-monotonicity occurs: higher performance may not only indicate greater luck, but also lower expected merit or quality. The semi-strong version is illustrated by examining the decoupling of citations of academic publications and their impact, illuminating when higher citations indicate lower quality. We conclude by discussing the broader implications of a normative theory of luck, emphasizing strategies to address situations where people mistake luck for skill.

And the award goes to…the Matthew Effect: Examining external status as a predictor of productivity and opportunity

It has been suggested that increased status that comes from being an award winner can generate enduring advantages that compound over one's career via the Matthew Effect. However, research in this area has yielded conflicting results and has been unable to isolate the unique effect of status on career outcomes from the positive endogenous characteristics of award winners. In the current research, we attempt to address previous research limitations and examine if winning an award is associated with career outcomes (i.e., opportunities and productivity) irrespective of individual productivity levels prior to receiving an award. We examined our research questions using observational data of National Hockey League (NHL) league championship winners and non-winners (N = 427). By using a team award and several different analytic approaches we were able to examine the unique effects of affiliation-based external status, generated from an award win, on career outcomes. Our results generally show support for the Matthew Effect and suggest that affiliation-based external status, achieved by an award win, provides access to increased opportunities, which ultimately results in more productivity. We discuss the importance of incorporating opportunity and investigating its role in the cumulative advantage process and implications of the results.

Autonomous, bidding, credible, decentralized, ethical, and funded (ABCDEF) publishing

Scientists write research articles, process ethics reviews, evaluate proposals and research, and seek funding. Several strategies have been proposed to optimize these operations and to decentralize access to research resources and opportunities. For instance, we previously proposed the trinity review method, combining registered reports with financing and research ethics assessments. However, previously proposed systems have a number of shortcomings, including how to implement them, e.g., who manages them, how incentives for reviewers are paid, etc. Various solutions have been proposed to address these issues, employing methods based on blockchain technologies, called "decentralized science (DeSci)". Decentralized approaches that exploit these developments offer potentially profound improvements to the troubled scientific ecosystem. Here, we propose a system that integrates ethics reviews, peer reviews, and funding in a decentralized manner, based on Web3 technology. This new method, named ABCDEF publishing, would enhance the speed, fairness, and transparency of scientific research and publishing.

Ranking versus rating in peer review of research grant applications

The allocation of public funds for research has been predominantly based on peer review where reviewers are asked to rate an application on some form of ordinal scale from poor to excellent. Poor reliability and bias of peer review rating has led funding agencies to experiment with different approaches to assess applications. In this study, we compared the reliability and potential sources of bias associated with application rating with those of application ranking in 3,156 applications to the Canadian Institutes of Health Research. Ranking was more reliable than rating and less susceptible to the characteristics of the review panel, such as level of expertise and experience, for both reliability and potential sources of bias. However, both rating and ranking penalized early career investigators and favoured older applicants. Sex bias was only evident for rating and only when the applicant's H-index was at the lower end of the H-index distribution. We conclude that when compared to rating, ranking provides a more reliable assessment of the quality of research applications, is not as influenced by reviewer expertise or experience, and is associated with fewer sources of bias. Research funding agencies should consider adopting ranking methods to improve the quality of funding decisions in health research.

Targeted, actionable and fair: Reviewer reports as feedback and its effect on ECR career choices

Previous studies of the use of peer review for the allocation of competitive funding agencies have concentrated on questions of efficiency and how to make the 'best' decision, by ensuring that successful applicants are also the more productive or visible in the long term. This paper examines the components of feedback received from an unsuccessful grant application, is associated with motivating applicants career decisions to persist (reapply for funding at T1), or to switch (not to reapply, or else leave academia). This study combined data from interviews with unsuccessful ECR applicants (n = 19) to The Wellcome Trust 2009-19, and manual coding of reviewer comments received by applicants (n = 81). All applicants received feedback on their application at T0 with a large proportion of unsuccessful applicants reapplying for funding at T1. Here, peer-review-comments-as-feedback sends signals to applicants to encourage them to persist (continue) or switch (not continue) even when the initial application has failed. Feedback associated by unsuccessful applicants as motivating their decision to resubmit had three characteristics: actionable; targeted; and fair. The results lead to identification of standards of feedback for funding agencies and peer-reviewers to promote when providing reviewer feedback to applicants as part of their peer review process. The provision of quality reviewer-reports-as-feedback to applicants, ensures that peer review acts as a participatory research governance tool focused on supporting the development of individuals and their future research plans.

Ranking mobility and impact inequality in early academic careers

How difficult is it for an early career academic to climb the ranks of their discipline? We tackle this question with a comprehensive bibliometric analysis of 57 disciplines, examining the publications of more than 5 million authors whose careers started between 1986 and 2008. We calibrate a simple random walk model over historical data of ranking mobility, which we use to 1) identify which strata of academic impact rankings are the most/least mobile and 2) study the temporal evolution of mobility. By focusing our analysis on cohorts of authors starting their careers in the same year, we find that ranking mobility is remarkably low for the top- and bottom-ranked authors and that this excess of stability persists throughout the entire period of our analysis. We further observe that mobility of impact rankings has increased over time, and that such rise has been accompanied by a decline of impact inequality, which is consistent with the negative correlation that we observe between such two quantities. These findings provide clarity on the opportunities of new scholars entering the academic community, with implications for academic policymaking.

The top 100 cited articles in menstrual health among adolescent girls: a citation analysis

BACKGROUND

Menstrual health is an important public health concern where it is still considered a taboo, and adolescent girls often lack knowledge about menstrual health, face limited access to sanitation facilities, and struggle with the affordability of sanitary materials. Every year numerous articles are published; however, only a few of them would be influential in the evolution of a particular field. The number of citations received by an article serves as a quality factor for the impact of the article in a particular field. Citation analysis analyses the relationship between citations received by articles. From the literature search, no citation analysis was conducted on menstrual health. Hence the objective of the study was to identify the articles which received hundred or more citations and also to identify the leading countries, journals, study designs, and departments conducting research on menstrual health.

METHODS

Citation analysis was done with search terms pertaining to adolescent and menstrual health using Google Scholar as a database in Publish or Perish software. The articles retrieved were exported to Microsoft Excel. Articles that received a hundred or more citations were screened for the type of article, department, and country where the study was conducted. A descriptive analysis of the hundred or more cited articles was done in Microsoft Excel.

RESULTS

A total of 982 articles pertaining to menstrual health among adolescent girls were retrieved. There were hundred articles with hundred and more citations pertaining to the menstrual health of adolescent girls. Cross-sectional study design, Obstetrics and Gynaecology department, India and USA countries, and PLOS ONE journal had the most citations in research on menstrual health among adolescent girls. The top ten articles were on menorrhagia, menstrual hygiene practices, Water, Sanitation and hygiene (WASH), stigma on menstruation, and education on menstrual health.

CONCLUSION

The hundred cited articles on menstrual health among adolescent girls were mainly from high-income countries and were of more observational in nature than interventional. Thus, highlighting the need to strengthen experimental studies on the menstrual health of adolescent girls in Lower-middle-income countries.

Data, measurement and empirical methods in the science of science

The advent of large-scale datasets that trace the workings of science has encouraged researchers from many different disciplinary backgrounds to turn scientific methods into science itself, cultivating a rapidly expanding 'science of science'. This Review considers this growing, multidisciplinary literature through the lens of data, measurement and empirical methods. We discuss the purposes, strengths and limitations of major empirical approaches, seeking to increase understanding of the field's diverse methodologies and expand researchers' toolkits. Overall, new empirical developments provide enormous capacity to test traditional beliefs and conceptual frameworks about science, discover factors associated with scientific productivity, predict scientific outcomes and design policies that facilitate scientific progress.

Non-White scientists appear on fewer editorial boards, spend more time under review, and receive fewer citations

Disparities continue to pose major challenges in various aspects of science. One such aspect is editorial board composition, which has been shown to exhibit racial and geographical disparities. However, the literature on this subject lacks longitudinal studies quantifying the degree to which the racial composition of editors reflects that of scientists. Other aspects that may exhibit racial disparities include the time spent between the submission and acceptance of a manuscript and the number of citations a paper receives relative to textually similar papers, but these have not been studied to date. To fill this gap, we compile a dataset of 1,000,000 papers published between 2001 and 2020 by six publishers, while identifying the handling editor of each paper. Using this dataset, we show that most countries in Asia, Africa, and South America (where the majority of the population is ethnically non-White) have fewer editors than would be expected based on their share of authorship. Focusing on US-based scientists reveals Black as the most underrepresented race. In terms of acceptance delay, we find, again, that papers from Asia, Africa, and South America spend more time compared to other papers published in the same journal and the same year. Regression analysis of US-based papers reveals that Black authors suffer from the greatest delay. Finally, by analyzing citation rates of US-based papers, we find that Black and Hispanic scientists receive significantly fewer citations compared to White ones doing similar research. Taken together, these findings highlight significant challenges facing non-White scientists.

Rethinking academia in a time of climate crisis

Addressing the climate crisis requires radical and urgent action at all levels of society. Universities are ideally positioned to lead such action but are largely failing to do so. At the same time, many academic scientists find their work impeded by bureaucracy, excessive competitiveness, and a loss of academic freedom. Here, drawing on the framework of "Doughnut Economics," developed by Kate Raworth, we suggest seven new principles for rethinking the norms of scientific practice. Based on these, we propose a call to action, and encourage academics to take concrete steps towards the creation of a flourishing scientific enterprise that is fit for the challenges of the 21st century.

The top 100 most-cited articles on adult spinal deformity: The most popular topics are still sagittal plane parameters and complications

Purpose

This study aimed to summarize the characteristics of the 100 most-cited articles on adult spinal deformity (ASD) and to analyze past and current research hotspots and trends.

Methods

Literature searches (from inception to 28 April 2022) using Web of Science databases were conducted to identify ASD-related articles. The top 100 most-cited articles were collected for further analysis. Meanwhile, author keywords from articles published in the last 5 years were selected for further analysis.

Results

The top 100 most-cited articles on ASD were selected from 3,354 papers. The publication year ranged from 1979 to 2017, and all papers were written in English. The citation count among them ranged from 100 to 1,145, and the mean citation number was 215.2. The foremost productive first author was Schwab F. University of Washington had the largest number of publications. The United States of America had the largest number of published articles (n = 84) in this field. Spine was the most popular journal. Complications were the most studied themes. The visualization analysis of author keywords from the literature in the recent 5 years showed that complications, sagittal plane parameters, and surgical techniques are still the research hotspots, and minimally invasive surgery will continue to develop rapidly.

Conclusion

Based on a comparative analysis of the results of bibliometric and visualization, complications and sagittal plane parameters are still the major topics of research at present and even later, and minimally invasive surgery has a growth trend in this field of ASD.

Critical reflections of postgraduate researchers on a collaborative interdisciplinary research project

By employing a retrospective collaborative autoethnographic approach, this work aims to better understand how an interdisciplinary context shaped the authors' experiences of British academia during their Ph.D research. The authors bring together their individual observations and experiences to collectively interrogate and critically reflect on their position as postgraduate researchers (PGRs) on a collaborative interdisciplinary research project. These reflections are taken as a lens through which to interrogate the contemporary British university. Pre-existing tensions within the academy are characterised as 'asymmetries' along dimensions of risk, disciplinary hierarchy, and knowledge. It is argued that the authors' experience of uncertainty and precarity as junior academics stems principally from pre-existing structures within British academia, rather than the interdisciplinary environment in which they were immersed. By emphasising the role of the successfully trained doctoral candidate as an outcome itself, it is argued that indicators of success can be reframed, shifting the power asymmetry to place greater value on PGRs within the neoliberal academy. Highlighting the ambiguity of their convergent and divergent personal experiences, the authors suggest there is a need for a greater focus on the contested role of the PGR within the contemporary university system.

Continued collaboration shortens the transition period of scientists who move to another institution

Scientific collaboration plays a significant role in scientists' research performance. When scientists move from one institution to another and leave the team they belong to or lead, they may continue collaborating with the former team because engaging in or building a new team takes time. In this study, we collected data from the Open Researcher and Contributor ID (ORCID) website on 2,922 scientists who published first-tier journal papers defined by the Chinese Academy of Science (CAS) before they moved to a new institution. By applying a Poisson regression model to the dataset, we explored the correlation between continued collaboration and the transition period after scientists moved, which is defined as the time span between the year of the move and the year when they published their first top-tier journal paper after moving. Our findings indicated that: (1) continued collaboration significantly shortens the transition period by 27.2%; (2) continued collaboration significantly shortens the transition period of senior scientists to a larger extent than that of junior scientists; (3) continued collaboration significantly shortens the transition period of social scientists to a larger extent than that of natural scientists; (4) the transition period is shorter after moves for scientists with higher inherent potential; and (5) there is no evidence that the transition period is associated with culture-related differences between the origin country and the destination country after the move, or whether they had lived in the destination country before.

From Innovation to Intrapreneurship: Fostering academic success via the GridlockED project and innovation fund

Background

Funding for educational innovations is increasingly scarce in academic medicine. While there is some funding for medical education research, this is often for discovery or application work, and there are few avenues for those with a heavy innovation focus to fund early work.

Objective of the Innovation

The objective was to develop an intrapreneurial unit focused on medical education projects and scholarship.

Development Process and Implementation

The GridlockED and TriagED games are educational or serious games that seek to teach health care learners about emergency medicine processes. Both games were cocreated with learners and brought to market in the past 3 years. All of the proceeds from the sales of these games have been accrued over time to create a new innovation fund. This fund seeks to support trainees and early career educators in their medical education projects.

Outcomes

Sales for GridlockED began in March 2018 and the TriagED began in November 2019. In the first year, sales for GridlockED yielded a total of $9,534. After 18 months of sales, the fund has accrued a total of $14,530. The fund has helped finance the development of new games. Additionally, the fund awarded two internal $500 Kickstarter grants to assist with evaluating and improving two local education projects. The GridlockED and TriagED games have also spurred multiple academic opportunities for junior educators interested in this domain: five workshops, eight conference abstracts, two peer-reviewed papers, and two research protocols are being developed.

Conclusions

The GridlockED and TriagED games represent a new academically oriented, intrapreneurial approach to medical education work.

Sticky steps and the gender gap: how thoughtful practices could help keep caregivers in science

Many fewer women than men hold senior academic positions, a widely recognized and increasing problem. Our goal is to identify effective and feasible solutions. We begin by providing an in-depth assessment of the drivers of this gender inequity. In our synthesis of existing data, we provide many lines of evidence highlighting caregiving as a primary main factor. This is not a 'new' insight per se, but a point worth repeating that we back up by a strong and synthetic body of recent data. We also believe that our analysis provides a step forward in tackling a complex issue. We then develop a more detailed understanding of the challenges academic caregivers face and discuss whether and why it is important to keep caregivers in science. We find that the attrition due to caregiving should not be seen as a factor but rather as a process with multiple 'sticky steps' that eventually drive caregivers out of science-which, as we argue, is partly also good news. Indeed, it is here that we believe actions could be taken that would have a real impact: for example, one could effectively increase and expand upon current funding practices that focus on caregiver career advancement.

Systemic racial disparities in funding rates at the National Science Foundation

Concerns about systemic racism at academic and research institutions have increased over the past decade. Here, we investigate data from the National Science Foundation (NSF), a major funder of research in the United States, and find evidence for pervasive racial disparities. In particular, white principal investigators (PIs) are consistently funded at higher rates than most non-white PIs. Funding rates for white PIs have also been increasing relative to annual overall rates with time. Moreover, disparities occur across all disciplinary directorates within the NSF and are greater for research proposals. The distributions of average external review scores also exhibit systematic offsets based on PI race. Similar patterns have been described in other research funding bodies, suggesting that racial disparities are widespread. The prevalence and persistence of these racial disparities in funding have cascading impacts that perpetuate a cumulative advantage to white PIs across all of science, technology, engineering, and mathematics.

Relationship between early-career collaboration among researchers and future funding success in Japanese academia

Academia is becoming more and more competitive, especially for young scientists, so it is important to understand the factors that affect success in academic careers. To survive in academia, it is crucial to obtain research funding. Previous studies have investigated factors that affect the funding success of researchers. In this paper, we focus on research collaboration structure as a factor affecting funding success. More specifically, we investigate the effects of participation in joint research projects, number of joint research projects, and centrality in the collaborative network on the future funding success of junior researchers in Japan. Our results show that participation in joint research projects and the number of such projects significantly affect the future funding success of junior researchers. Furthermore, the median number of years of funding received by researchers involved in joint research projects was found to be about 1.5 times greater than that of researchers not involved in joint research projects, and the average amount of research funding received after 10 years is about 2–4 times more, suggesting that researchers with collaboration ties with other researchers in the early stages of their career tend to be more successful in the future.

Getting funded in a highly fluctuating environment: Shifting from excellence to luck and timing

Recent data highlights the presence of luck in research grant allocations, where most vulnerable are early-career researchers. The national research funding contributes typically the greatest share of total research funding in a given country, fulfilling simultaneously the roles of promoting excellence in science, and most importantly, development of the careers of young generation of scientists. Yet, there is limited supply of studies that have investigated how do early-career researchers stand compared to advanced-career level researchers in case of a national research grant system. We analyzed the Estonian national highly competitive research grant funding across different fields of research for a ten-year-period between 2013-2022, including all the awarded grants for this period (845 grants, 658 individual principal investigators, PI). The analysis was conducted separately for early-career and advanced-career researchers. We aimed to investigate how the age, scientific productivity and the previous grant success of the PI vary across a national research system, by comparing early- and advanced-career researchers. The annual grant success rates varied between 14% and 28%, and within the discipline the success rate fluctuated across years even between 0-67%. The year-to-year fluctuations in grant success were stronger for early-career researchers. The study highlights how the seniority does not automatically deliver better research performance, at some fields, younger PIs outperform older cohorts. Also, as the size of the available annual grants fluctuates remarkably, early-career researchers are most vulnerable as they can apply for the starting grant only within a limited "time window".

A guide to megastudies

How can behavioral insights best be leveraged to solve pressing policy challenges? Because research studies are typically designed to test the validity of a particular idea, surprisingly little is known about the relative efficacy of different approaches to changing behavior in any given policy context. We discuss megastudies as a research approach that can surmount this and other obstacles to developing optimal behaviorally informed policy interventions. We define a megastudy as "a massive field experiment in which many different treatments are tested synchronously in one large sample using a common, objectively measured outcome." We summarize this apples-to-apples approach to research and lay out recommendations, limitations, and promising future directions for scholars who might want to conduct or evaluate megastudies.

Reckoning With Our Crisis: An Agenda for the Field of Social and Personality Psychology

The replication crisis and credibility revolution in the 2010s brought a wave of doubts about the credibility of social and personality psychology. We argue that as a field, we must reckon with the concerns brought to light during this critical decade. How the field responds to this crisis will reveal our commitment to self-correction. If we do not take the steps necessary to address our problems and simply declare the crisis to be over or the problems to be fixed without evidence, we risk further undermining our credibility. To fully reckon with this crisis, we must empirically assess the state of the field to take stock of how credible our science actually is and whether it is improving. We propose an agenda for metascientific research, and we review approaches to empirically evaluate and track where we are as a field (e.g., analyzing the published literature, surveying researchers). We describe one such project (Surveying the Past and Present State of Published Studies in Social and Personality Psychology) underway in our research group. Empirical evidence about the state of our field is necessary if we are to take self-correction seriously and if we hope to avert future crises.

How scientists interpret and address funding criteria: value creation and undesirable side effects

Scientists and funding bodies are interdependent actors involved in an ongoing two-way signalling interaction; however, we lack insight on the social mechanisms underpinning this interaction. To address this issue, we examine how successfully funded scientists interpret and address criteria set by the funding body to maximise their chances of funding success. We also consider the possible adverse side effects that can arise from scientists' competitive efforts to address these criteria. Our findings identify a portfolio of funding criteria-research feasibility, research alignment and team credentials-that scientists address when preparing grant applications. Effectively addressing these criteria enhances the prospects of funding success and value creation. However, we also find that scientists can over-address funding criteria, which is counterproductive and yields undesirable side effects. Our research therefore makes an important distinction between the possibilities for value creation and the value creation frictions that can unintentionally arise based on how grant-submitting scientists interpret and address the criteria signalled by the funding body. Our research has implications for policymakers, funding bodies and scientists which we also discuss.

Bibliometric Approach to Evaluating the Impact of a Building Interdisciplinary Research Careers in Women's Health K12 Research Career Development Program

Background: Mentored research career development programs are excellent training opportunities for junior faculty/early-stage investigators to transition into independent research careers. However, there is limited evidence that provides guidance on best practices for measuring the impact and reach of these programs, both for individual Scholars and the program as a whole. This article evaluates both the individual and overall impact of the Building Interdisciplinary Research Careers in Women's Health (BIRCWH) National Institutes of Health research career development award at the University of Minnesota. Materials and Methods: BIRCWH Scholars (n = 16) and a comparison group (n = 17) were evaluated on traditional metrics (e.g., publications, grant funding) in addition to bibliometrics (e.g., network growth, interdisciplinary collaborations, international reach, policy impact). Results: Traditional metric findings showed that BIRCWH Scholars had significantly more publications from pre- to post-BIRCWH experience than the comparison group and more grant funding. Bibliometric findings showed exponential network growth, interdisciplinary collaborations, international citations, and policy impact from pre- to post-BIRCWH Scholar experience. Conclusion: Findings from this evaluation have potential important implications. At the Scholar level, the results can be used to provide evidence of research impact in materials developed for merit review and promotion as well as in job and research grant applications. At the program level, the results can be used at the institutional level to gain broad administrative support and leverage additional funds for program activities and for evidence of program success for continuation funding from federal agencies.

Fund behavioral science like the frameworks we endorse: the case for increased funding of preliminary studies by the National Institutes of Health

Innovative, groundbreaking science relies upon preliminary studies (aka pilot, feasibility, proof-of-concept). In the behavioral sciences, almost every large-scale intervention is supported by a series of one or more rigorously conducted preliminary studies. The importance of preliminary studies was established by the National Institutes of Health (NIH) in 2014/2015 in two translational science frameworks (NIH Stage and ORBIT models). These frameworks outline the essential role preliminary studies play in developing the next generation of evidence-based behavioral prevention and treatment interventions. Data produced from preliminary studies are essential to secure funding from the NIH's most widely used grant mechanism for large-scale clinical trials, namely the R01. Yet, despite their unquestionable importance, the resources available for behavioral scientists to conduct rigorous preliminary studies are limited. In this commentary, we discuss ways the existing funding structure at the NIH, despite its clear reliance upon high-quality preliminary studies, inadvertently discourages and disincentivizes their pursuit by systematically underfunding them. We outline how multiple complementary and pragmatic steps via a small reinvestment of funds from larger trials could result in a large increase in funding for smaller preliminary studies. We make the case such a reinvestment has the potential to increase innovative science, increase the number of investigators currently funded, and would yield lasting benefits for behavioral science and scientists alike.