1
|
Recio-Saucedo A, Crane K, Meadmore K, Fackrell K, Church H, Fraser S, Blatch-Jones A. What works for peer review and decision-making in research funding: a realist synthesis. Res Integr Peer Rev 2022; 7:2. [PMID: 35246264 PMCID: PMC8894828 DOI: 10.1186/s41073-022-00120-2] [Citation(s) in RCA: 12] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/05/2021] [Accepted: 02/01/2022] [Indexed: 11/17/2022] Open
Abstract
INTRODUCTION Allocation of research funds relies on peer review to support funding decisions, and these processes can be susceptible to biases and inefficiencies. The aim of this work was to determine which past interventions to peer review and decision-making have worked to improve research funding practices, how they worked, and for whom. METHODS Realist synthesis of peer-review publications and grey literature reporting interventions in peer review for research funding. RESULTS We analysed 96 publications and 36 website sources. Sixty publications enabled us to extract stakeholder-specific context-mechanism-outcomes configurations (CMOCs) for 50 interventions, which formed the basis of our synthesis. Shorter applications, reviewer and applicant training, virtual funding panels, enhanced decision models, institutional submission quotas, applicant training in peer review and grant-writing reduced interrater variability, increased relevance of funded research, reduced time taken to write and review applications, promoted increased investment into innovation, and lowered cost of panels. CONCLUSIONS Reports of 50 interventions in different areas of peer review provide useful guidance on ways of solving common issues with the peer review process. Evidence of the broader impact of these interventions on the research ecosystem is still needed, and future research should aim to identify processes that consistently work to improve peer review across funders and research contexts.
Collapse
Affiliation(s)
- Alejandra Recio-Saucedo
- Wessex Institute, National Institute of Health Research Evaluation, Trials and Studies Coordinating Centre, University of Southampton, Alpha House, Enterprise Road, Southampton, Southampton, SO16 7NS UK
| | - Ksenia Crane
- Wessex Institute, National Institute of Health Research Evaluation, Trials and Studies Coordinating Centre, University of Southampton, Alpha House, Enterprise Road, Southampton, Southampton, SO16 7NS UK
| | - Katie Meadmore
- Wessex Institute, National Institute of Health Research Evaluation, Trials and Studies Coordinating Centre, University of Southampton, Alpha House, Enterprise Road, Southampton, Southampton, SO16 7NS UK
| | - Kathryn Fackrell
- Wessex Institute, National Institute of Health Research Evaluation, Trials and Studies Coordinating Centre, University of Southampton, Alpha House, Enterprise Road, Southampton, Southampton, SO16 7NS UK
| | - Hazel Church
- Wessex Institute, National Institute of Health Research Evaluation, Trials and Studies Coordinating Centre, University of Southampton, Alpha House, Enterprise Road, Southampton, Southampton, SO16 7NS UK
| | - Simon Fraser
- Wessex Institute, National Institute of Health Research Evaluation, Trials and Studies Coordinating Centre, University of Southampton, Alpha House, Enterprise Road, Southampton, Southampton, SO16 7NS UK
- School of Primary Care, Population Sciences and Medical Education, Faculty of Medicine, University of Southampton, Southampton, SO17 1BJ UK
| | - Amanda Blatch-Jones
- Wessex Institute, National Institute of Health Research Evaluation, Trials and Studies Coordinating Centre, University of Southampton, Alpha House, Enterprise Road, Southampton, Southampton, SO16 7NS UK
| |
Collapse
|
3
|
Győrffy B, Nagy AM, Herman P, Török Á. Factors influencing the scientific performance of Momentum grant holders: an evaluation of the first 117 research groups. Scientometrics 2018; 117:409-426. [PMID: 30220748 PMCID: PMC6132953 DOI: 10.1007/s11192-018-2852-1] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2018] [Indexed: 11/28/2022]
Abstract
The Momentum program launched in 2009 provides funding of up to 1 million Euro to establish new, independent research groups at Hungarian academic institutions. Here, our aim was to determine factors associated with the scientific output of these research groups. Publication data were downloaded from the Hungarian Scientific Work Archive (www.mtmt.hu), impact factor data were obtained from Thomson Reuters (jcr.incites.thomsonreuters.com), and journal ranks were extracted from the Scimago Journal Rank database (www.scimagojr.com). Investigated input features for each grant holder include gender, degree, targeted category, international mobility, international grants, number of publications, total number of citations, H-index, best publications, impact factors in the last 2 years, and assessment scores provided by the experts. Evaluated performance indicators include cumulative impact factor, number of D1 publications, and number of first/last author D1 publications during the grant running time. Grant holders’ publication output increased by 23 and 52% for life sciences and material sciences researchers. Scientific performance was independent from gender, degree, international grants, category applied for, and citations received for the best pre-grant publication. Those with international mobility had significantly lower scientific output (yearly impact factor, number of D1 publications, number of first/last author publications). Scores received from grant review experts were independent from later publication activity. The strongest correlations were observed between scientific output and total number of citations, H-index, and impact factor in the last 2 years pre-grant. In summary, group leaders with a dynamic publication track record were able to attain the most additional momentum. Our results can help accelerate and improve future grant review processes.
Collapse
Affiliation(s)
- Balázs Győrffy
- 1MTA TTK Lendület Cancer Biomarker Research Group, Institute of Enzymology, Hungarian Academy of Sciences, Magyar Tudósok körútja 2, Budapest, 1117 Hungary.,22nd Department of Pediatrics, Semmelweis University, Tűzoltó utca 7-9., Budapest, 1094 Hungary
| | - Andrea Magda Nagy
- 3Department of Economics, University of Pannonia, Egyetem u. 10, Veszprém, 8200 Hungary
| | - Péter Herman
- 1MTA TTK Lendület Cancer Biomarker Research Group, Institute of Enzymology, Hungarian Academy of Sciences, Magyar Tudósok körútja 2, Budapest, 1117 Hungary
| | - Ádám Török
- 4Department of International Economics, University of Pannonia, Egyetem u. 10, Veszprém, 8200 Hungary.,5Department of Economics, Budapest University of Technology and Economics, Magyar Tudósok körútja 4, Budapest, 1117 Hungary
| |
Collapse
|