How significant are the public dimensions of faculty work in review, promotion and tenure documents?

A dynamic framework for making sense of partnerships between universities and informal education providers

We present a conceptual framework rooted in the practices and experiences of nine collaborative partnerships between universities and informal science education organizations. Our analysis and resulting framework aim to support those brokering, designing, and evaluating partnerships in making sense of the dynamism of interorganizational efforts to collaborate in achieving broader impacts of research. We highlight the critical role of brokers in modulating attentions between interorganizational, intraorganizational, and networked dimensions of their collaborations to maintain partnership health and continue progress toward shared goals.

Barriers to Meaningful Participatory Mental Health Services Research and Priority Next Steps: Findings From a National Survey

OBJECTIVE

A growing consensus has emerged regarding the importance of stakeholder involvement in mental health services research. To identify barriers to and the extent of stakeholder involvement in participatory research, the authors undertook a mixed-methods study of researchers and community members who reported participation in such research.

METHODS

Eight consultative focus groups were conducted with diverse groups of stakeholders in mental health services research (N=51 unique participants, mostly service users), followed by a survey of service users, family members, community providers, and researchers (N=98) with participatory research experience. Focus groups helped identify facilitators and barriers to meaningful research collaboration, which were operationalized in the national survey. Participants were also asked about high-priority next steps.

RESULTS

The barrier most strongly endorsed as a large or very large problem in the field was lack of funding for stakeholder-led mental health services research (76%), followed by lack of researcher training in participatory methods (74%) and insufficiently diverse backgrounds among stakeholders (69%). The two most frequently identified high-priority next steps were ensuring training and continuing education for researchers and stakeholders (33%) and authentically centering lived experience and reducing tokenism in research (26%).

CONCLUSIONS

These findings suggest a need for increased attention to and investment in the development, implementation, and sustainment of participatory methods that prioritize collaboration with direct stakeholders, particularly service users, in U.S. mental health services research. The findings also underscore the presence and potentially important role of researchers who dually identify as service users and actively contribute a broader orientation from the service user-survivor movement.

Viewing research assessment, the academic reward system, and academic publishing through the power/knowledge lens of Foucault

The academic research assessment system, the academic reward system, and the academic publishing system are interrelated mechanisms that facilitate the scholarly production of knowledge. This article considers these systems using a Foucauldian lens to examine the power/knowledge relationships found within and through these systems. A brief description of the various systems is introduced followed by examples of instances where Foucault's power, knowledge, discourse, and power/knowledge concepts are useful to provide a broader understanding of the norms and rules associated with each system, how these systems form a network of power relationships that reinforce and shape one another.

Analysis of Teaching-Related Criteria Within Promotion and Tenure Documents from US Pharmacy Schools

OBJECTIVE

To describe teaching-related criteria within promotion and tenure (PT) guidance documents from US-based colleges/schools of pharmacy.

METHODS

PT guidance documents were retrieved from college/school websites or via electronic mail. Institutional characteristics were compiled using available online data. Using qualitative content analysis, a systematic review of PT guidance documents was conducted to identify how teaching and teaching excellence were considered in decision of promotion and/or tenure at each institution.

RESULTS

PT guidance documents were analyzed from 121 (85%) colleges/schools of pharmacy. Of these, 40% included a requirement that faculty must achieve excellence in teaching for promotion and/or tenure, though excellence was infrequently defined (14% of colleges/schools). Criteria specific to didactic teaching were most frequently included (94% of institutions). Criteria specific to experiential (50%), graduate student (48%), postgraduate (41%), and interprofessional (13%) teaching were less frequently included. Institutions frequently required student (58%) and peer (50%) evaluations of teaching to be considered in PT decisions. Most institutions acknowledged many teaching accomplishments as examples indicating teaching success rather than strictly requiring specific criteria to be fulfilled.

CONCLUSION

Teaching-related criteria within PT criteria of colleges/schools of pharmacy often lack clear guidance regarding quantitative or qualitative requirements for advancement. This lack of clearly specified requirements may result in faculty members' inability to self-assess for readiness for promotion and inconsistent application of criteria in PT decisions by review committees and administrators.

Transforming the field: the role of academic health centers in promoting and sustaining equity based community engaged research

Community-based participatory research (CBPR) and community engaged research (CEnR) are key to promoting community and patient engagement in actionable evidence-based strategies to improve research for health equity. Rapid growth of CBPR/CEnR research projects have led to the broad adoption of partnering principles in community-academic partnerships and among some health and academic organizations. Yet, transformation of principles into best practices that foster trust, shared power, and equity outcomes still remain fragmented, are dependent on individuals with long term projects, or are non-existent. This paper describes how we designed our Engage for Equity PLUS intervention that leverages the leadership and membership of champion teams (including community-engaged faculty, community partners and patient advocates) to improve organizational policies and practices to support equity based CBPR/CEnR. This article describes the feasibility and preliminary findings from engaging champion teams from three very different academic health centers. We reflect on the learnings from Engage for Equity PLUS; the adaptation of the intervention design and implementation, including the development of a new institutional assessment using mixed research methods; and our organizational theory of change. In summary, our design and preliminary data from the three academic health centers provide support for new attention to the role of institutional practices and processes needed to sustain equity-based patient and community-engaged research and CBPR and transform the field.

Levers of Change: How to Help Build the Public Health Workforce of the Future

CONTEXT

Public health leaders are working to rebuild the US public health workforce. Master of Public Health (MPH) programs have a stake in this, given their role in educating and training public health practitioners. Over the last 10 years, MPH programs have implemented changes to program structure, content, and approach, but workforce gaps persist.

OBJECTIVE

This study sought to explore the factors that inform and influence MPH program design and changes they make in order to elucidate how MPH programs may be further engaged to help address current and future public health workforce needs.

DESIGN

Sequential mixed-methods study.

SETTING

US MPH programs accredited by the Council on Education for Public Health (CEPH), and applicants approved to seek accreditation.

PARTICIPANTS

In total, 115 representatives representing at least 43% of the 215 accredited/applicant MPH programs in the United States.

MAIN OUTCOME MEASURES

Factors that inform and influence programmatic and curricular changes within MPH programs.

RESULTS

The shifts that MPH programs have made to program focus and the approaches used to support student competence development are influenced by individual, programmatic, institutional, and national factors, including faculty and staff background, access to resources, program team/faculty culture, access to resources, program placement, university priorities, and national policies. Most influential in catalyzing changes made by MPH programs between 2015 and 2020 were CEPH MPH accreditation standards, feedback from interested parties, learning best practices, university initiatives, and access to resources including funding and faculty. Identified factors served as facilitators and/or as barriers, depending on the context.

CONCLUSIONS

There are multiple levers at different levels that may be utilized by national public health leaders, university administrators, and program constituents to effect change within MPH programs, helping them to be even better positioned to help address public health workforce needs of today and tomorrow.

Universities claim to value community-engaged scholarship: So why do they discourage it?

With current crises of academic relevance and legitimacy, there is a need for epistemic equity inherent to community-engaged research. Scholars in science communication and science and technology studies have analyzed, advocated for, and conducted public engagement in pursuit of this goal. However, despite desires to celebrate public engagement, US academic institutions and organizations often present barriers to meaningful community-engaged research. From tenure and promotion requirements, to lack of recognition and resources, universities in the American academic landscape are not currently organized to support such work. In this article, we offer a conceptual framework to examine the complex structural dimensions of academic institutions that have systematically discouraged and devalued faculty participation in community-engaged scholarship. We outline four such structural dimensions: interrogating epistemic biases, neoliberalist tendencies, gendered norms, and colonial-racist defaults. Our goal is to illuminate processes that could inform interventions to bridge the gap between academic aspirations for community-engaged work and current actions in the academy that undermine it.

Could the altmetrics wave bring a flood of confusion for anatomists?

Altmetrics are non-traditional metrics that can capture downloads, social media shares, and other modern measures of research impact and reach. Despite most of the altmetrics literature focusing on evaluating the relationship between research outputs and academic impact/influence, the perceived and actual value of altmetrics among academicians remains nebulous and inconsistent. This work proposes that ambiguities surrounding the value and use of altmetrics may be explained by a multiplicity of altmetrics definitions communicated by journal publishers. A root cause analysis was initiated to compare altmetrics definitions between anatomy and medical education journal publishers' websites and to determine the comparability of the measurement and platform sources used for computing altmetrics values. A scoping content analysis of data from across eight publishers' websites revealed wide variability in definitions and heterogeneity among altmetrics measurement sources. The incongruencies among publishers' altmetrics definitions and their value demonstrate that publishers may be one of the root cause of ambiguity perpetuating confusion around the value and use of altmetrics. This review highlights the need to more deeply explore the root causes of altmetrics ambiguities within academia and makes a compelling argument for establishing a ubiquitous altmetrics definition that is concise, clear, and specific.

The disconnect between researcher ambitions and reality in achieving impact in the Earth & Environmental Sciences – author survey

Background: There is an increasing desire for research to provide solutions to the grand challenges facing our global society, such as those expressed in the UN SDGs (“real-world impact”). Herein, we undertook an author survey to understand how this desire influenced the choice of research topic, choice of journal, and preferred type of impact. Methods: We conducted a survey of authors who had published in >100 of our Earth & Environmental Science journals. The survey was sent to just under 60,000 authors and we received 2,695 responses (4% response rate).   Results: Respondents indicated that the majority of their research (74%) is currently concerned with addressing urgent global needs, whilst 90% of respondents indicated that their work either currently contributed to meeting real-world problems or that it would be a priority for them in the future; however, the impetus for this research focus seems to be altruistic researcher desire, rather than incentives or support from publishers, funders, or their institutions. Indeed, when contextualised within existing reward and incentive structures, respondents indicated that citations or downloads were more important to them than contributing to tackling real-world problems. Conclusions: At present, it seems that the laudable and necessary ambition of researchers in the Earth & Environmental Sciences to contribute to the tackling of real-world problems, such as those included in the UN SDGs, is seemingly being lost amidst the realities of being a researcher, owing to the prioritisation of other forms of impact, such as citations and downloads.

Team science criteria and processes for promotion and tenure of Health Science University Faculty

Although team science has expanded with far-reaching benefits, universities generally have not established criteria to recognize its value in faculty promotion and tenure. This paper recommends how institutions might weigh a faculty member's engagement in team science in the promotion and tenure process. Seventeen team science promotion and tenure criteria are recommended based on four sources - an evaluation framework, effectiveness metrics, collaborative influences, and authorship criteria. Suggestions are made for adaptation of the 17 criteria to committee guidelines, faculty team science portfolios, and the roles of individuals and institutions participating in large, cross-disciplinary research projects. Future research recommendations are advanced.

How institutional factors at US land-grant universities impact scientists' public scholarship

The call for public scholarship to emphasize the broader impacts of science has raised questions about how universities can support this work among their scientists. This study quantitatively assesses how institutional factors shape scientists' participation in public scholarship, a subset of public engagement focusing on scientists' involvement in public debate and democratic decision-making related to science policy. Based on a 2018 survey of scientists from 46 US land-grant universities (N = 6,242), hierarchical linear modeling results show that institutional factors, including tenure guidelines and the extent of government funding, play a minor role in influencing scientists' public scholarship participation. More importantly, scientists' perceptions of the university climate on support for engagement, including support from high-level administrators and for graduate students, are significant predictors of participation in public scholarship. Ultimately, these findings support the recommendation that universities should coordinate individual motivations with institutional missions to support a broader culture of public engagement.

Valuation regimes in academia: Researchers’ attitudes towards their diversity of activities and academic performance

Evaluation systems have been long criticized for abusing and misusing bibliometric indicators. This has created a culture by which academics are constantly exposing their daily work to the standards they are expected to perform. In this study, we investigate whether researchers’ own values and expectations are in line with the expectations of the evaluation system. We conduct a multiple case study of five departments in two Dutch universities to examine how they balance between their own valuation regimes and the evaluation schemes. For this, we combine curriculum analysis with a series of semi-structured interviews. We propose a model to study the diversity of academic activities and apply it to the multiple case study to understand how such diversity is shaped by discipline and career stage. We conclude that the observed misalignment is not only resulting from an abuse of metrics but also by a lack of tools to evaluate performance in a contextualized and adaptable way.

Recognition of knowledge translation practice in Canadian health sciences tenure and promotion: A content analysis of institutional policy documents

BACKGROUND AND OBJECTIVE

There has been growing emphasis on increasing impacts of academic health research by integrating research findings in healthcare. The concept of knowledge translation (KT) has been widely adopted in Canada to guide this work, although lack of recognition in tenure and promotion (T&P) structures have been identified as barrier to researchers undertaking KT. Our objective was to explore how KT is considered in institutional T&P documentation in Canadian academic health sciences.

METHODS

We conducted content analysis of T&P documents acquired from 19 purposively sampled research-intensive or largest regional Canadian institutions in 2020-2021. We coded text for four components of KT (synthesis, dissemination, exchange, application). We identified clusters of related groups of documents interpreted together within the same institution. We summarized manifest KT content with descriptive statistics and identified latent categories related to how KT is considered in T&P documentation.

RESULTS

We acquired 89 unique documents from 17 institutions that formed 48 document clusters. Most of the 1057 text segments were categorized as dissemination (n = 851, 81%), which was included in 47 document clusters (98%). 15 document clusters (31%) included all four KT categories, while one (2%) did not have any KT categories identified. We identified two latent categories: primarily implicit recognition of KT; and an overall lack of clarity on KT.

CONCLUSIONS

Our analysis of T&P documents from primarily research-intensive Canadian universities showed a lack of formal recognition for a comprehensive approach to KT and emphasis on traditional dissemination. We recommend that institutions explicitly and comprehensively consider KT in T&P and align documentation and procedures to reflect these values.

Retention, mobility, and successful transition to independence of health sciences postdocs

INTRODUCTION

Obtaining a tenure track faculty position (TTFP) after postdoctoral appointment (PDA) completion is considered an indicator of successful transition to independence (TTI). Whether cross-institutional mobility (CIM)-moving to a different institution from that of the PDA-contributes to TTI is unclear, as data evaluating retention and mobility is lacking. We tested the hypothesis that, for postdocs (PDs) at R1 institutions, CIM is a significant predictor of successful TTI defined as TTFP-status 3 years post-PDA.

MATERIALS AND METHODS

Using University of Pittsburgh data for health sciences PDs we tested the association of CIM at PDA completion (moved to a different institution (CIM = 1) or retained at Pitt (CIM = 0)) with TTFP-status 3 years post-PDA (TTFP, non-TTFP, or left faculty position) using multinomial logistic regression models.

RESULTS

Among all 622 Pitt PDs, 3-year retention in a faculty position at Pitt was 21%, while 14% had a faculty position outside of Pitt. Among the analytic sample of PDs with an academic career outcome during the study period (N = 238; 50% women, 8% underrepresented minorities (URM)), at baseline PDA completion 39% moved to a different institution (CIM = 1), and 61% remained at Pitt (CIM = 0) in any job type. Those with CIM = 1 had greater odds of having a TTFP at follow-up than those with CIM = 0 [adjusted OR (95% CI): 4.4 (2.1, 9.2)].

DISCUSSION

One fifth of Pitt PDs were retained by Pitt as faculty. While Pitt PDs were equally likely to get a faculty position whether they were retained at Pitt or left, those who left had greater odds of obtaining a TTFP. Future work with longer follow-up times, expanded markers of TTI, and samples from other R1 institutions is needed to better understand the reason for these results. This knowledge can lead to better support for the next generation of PDs as they successfully transition to faculty.

The failure of success: four lessons learned in five years of research on research integrity and research assessments

In the past 5 years, we captured the perspectives from a broad array of research stakeholders to better understand the impact that current approaches to success and research assessment may have on the integrity and the quality of research. Here, we translate our findings in four actions that are urgently needed to foster better research. First, we need to address core research structures to overcome systemic problems of the research enterprise; second, we must realign research assessments to value elements that advance and strengthen science; third, we need to remodel, diversify, and secure research careers; and finally, we need to unite and coordinate efforts for change.

The role of blogs and news sites in science communication during the COVID-19 pandemic

We present a brief review of literature related to blogs and news sites; our focus is on publications related to COVID-19. We primarily focus on the role of blogs and news sites in disseminating research on COVID-19 to the wider public, that is knowledge transfer channels. The review is for researchers and practitioners in scholarly communication and social media studies of science who would like to find out more about the role of blogs and news sites during the COVID-19 pandemic. From our review, we see that blogs and news sites are widely used as scholarly communication channels and are closely related to each other. That is, the same research might be reported in blogs and news sites at the same time. They both play a particular role in higher education and research systems, due to the increasing blogging and science communication activity of researchers and higher education institutions (HEIs). We conclude that these two media types have been playing an important role for a long time in disseminating research, which even increased during the COVID-19 pandemic. This can be verified, for example, through knowledge graphs on COVID-19 publications that contain a significant amount of scientific publications mentioned in blogs and news sites.

The disconnect between researcher ambitions and reality in achieving impact in the Earth & Environmental Sciences – author survey

Background: There is an increasing desire for research to provide solutions to the grand challenges facing our global society, such as those expressed in the UN SDGs (“real-world impact”). Herein, we undertook an author survey to understand how this desire influenced the choice of research topic, choice of journal, and preferred type of impact. Methods: We conducted a survey of authors who had published in >100 of our Earth & Environmental Science journals. The survey was sent to just under 60,000 authors and we received 2,695 responses (4% response rate).   Results: Respondents indicated that the majority of their research (74%) is currently concerned with addressing urgent global needs, whilst 90% of respondents indicated that their work either currently contributed to meeting real-world problems or that it would be a priority for them in the future; however, the impetus for this research focus seems to be altruistic researcher desire, rather than incentives or support from publishers, funders, or their institutions. Indeed, when contextualised within existing reward and incentive structures, respondents indicated that citations or downloads were more important to them than contributing to tackling real-world problems. Conclusions: At present, it seems that the laudable and necessary ambition of researchers in the Earth & Environmental Sciences to contribute to the tackling of real-world problems, such as those included in the UN SDGs, is seemingly being lost amidst the realities of being a researcher, owing to the prioritisation of other forms of impact, such as citations and downloads.

Public engagement: Faculty lived experiences and perspectives underscore barriers and a changing culture in academia

The idea of faculty engaging in meaningful dialogue with different publics instead of simply communicating their research to interested audiences has gradually morphed from a novel concept to a mainstay within most parts of the academy. Given the wide variety of public engagement modalities, it may be unsurprising that we still lack a comprehensive and granular understanding of factors that influence faculty willingness to engage with public audiences. Those nuances are not always captured by quantitative surveys that rely on pre-determined categories to assess scholars’ willingness to engage. While closed-ended categories are useful to examine which factors influence the willingness to engage more than others, it is unlikely that pre-determined categories comprehensively represent the range of factors that undermine or encourage engagement, including perceptual influences, institutional barriers, and scholars’ lived experiences. To gain insight into these individual perspectives and lived experiences, we conducted focus group discussions with faculty members at a large midwestern land-grant university in the United States. Our findings provide context to previous studies of public engagement and suggest four themes for future research. These themes affirm the persistence of institutional barriers to engaging with the public, particularly the expectations in the promotion process for tenure-track faculty. However, we also find a perception that junior faculty and graduate students are challenging the status quo by introducing a new wave of attention to public engagement. This finding suggests a “trickle-up” effect through junior faculty and graduate students expecting institutional support for public engagement. Our findings highlight the need to consider how both top-down factors such as institutional expectations and bottom-up factors such as graduate student interest shape faculty members’ decisions to participate in public engagement activities.

Closing the knowledge-action gap in conservation with open science

The knowledge-action gap in conservation science and practice occurs when research outputs do not result in actions to protect or restore biodiversity. Among the diverse and complex reasons for this gap, three barriers are fundamental: knowledge is often unavailable to practitioners and challenging to interpret or difficult to use or both. Problems of availability, interpretability, and useability are solvable with open science practices. We considered the benefits and challenges of three open science practices for use by conservation scientists and practitioners. First, open access publishing makes the scientific literature available to all. Second, open materials (detailed methods, data, code, and software) increase the transparency and use of research findings. Third, open education resources allow conservation scientists and practitioners to acquire the skills needed to use research outputs. The long-term adoption of open science practices would help researchers and practitioners achieve conservation goals more quickly and efficiently and reduce inequities in information sharing. However, short-term costs for individual researchers (insufficient institutional incentives to engage in open science and knowledge mobilization) remain a challenge. We caution against a passive approach to sharing that simply involves making information available. We advocate a proactive stance toward transparency, communication, collaboration, and capacity building that involves seeking out and engaging with potential users to maximize the environmental and societal impact of conservation science.

Challenges and solutions for the promotion of medical sciences faculty members in Iran: a systematic review

INTRODUCTION

The faculty promotion system is expected to benefit the faculty, institute, and profession and lead to the sustainable and comprehensive development. This present systematic review aims to investigate the challenges and solutions for the promotion of medical sciences faculty members in Iran.

METHOD

This study was a systematic review conducted by searching in PubMed, Scopus, Eric, Web of Science, Cochrane, SID, Magiran, and https://irandoc.ac.ir/line with Persian and English terms in the period from 2015 to 2020. Study selection and data extraction were performed independently by reviewers.

RESULTS

Thirteen articles were included. Challenges and solutions for the promotion of medical sciences faculty members were reviewed and grouped into five main categories: 1. The general regulations for the promotion of faculty members, 2. Cultural, disciplinary, and social activities, 3. Educational activities, 4. Research-technology activities, and 5. Scientific-executive activities.

CONCLUSION

Despite several modifications to regulations for the promotion of medical sciences faculty members in Iran, this process still encounters challenges because of its complex nature. This article provides tips to policymakers on regulations of promotion for educational activities.

Faculty appointment and promotion in Taiwan's medical schools, a systematic analysis

BACKGROUND

A rigorous faculty appointment and promotion (FAP) system is vital for the success of any academic institution. However, studies examining the FAP system in Asian universities are lacking. We surveyed the FAP policies of Taiwan's medical schools and identified an overreliance on the CJA score (manuscript Category, Journal quality, and Author order). The potential shortcomings of this metric and recommendations for refinement were discussed.

METHODS

We obtained the FAP documents from all 12 medical schools in Taiwan, and analyzed their use of traditional versus non-traditional criteria for FAP according to a published methodology. The influence of the journal impact factor (JIF) on the FAP process was quantified by comparing its relative weight between papers with two extreme JIFs. To better understand the research impact and international standing of each school, we utilized the public bibliographic database to rank universities by the number of papers, and the proportions of papers within the top 10% or 50% citation.

RESULTS

Compared with other countries, Taiwan's medical schools focus more on the quantifiable quality of the research, mostly using a "CJA" score that integrates the category, JIF or ranking, and authorship of a paper, with the JIF being the most influential factor. The CJA score for an article with a JIF of 20 can be up to three times the threshold for promotion to Assistant Professor. The emphasis on JIF is based on a presumed correlation between JIF and citation counts. However, our analysis shows that Taiwan's medical schools have lower-than-average citation counts despite a competitive rank in the number of publications.

CONCLUSIONS

The JIF plays an unrivaled role in determining the outcome of FAP in Taiwan's medical schools, mostly via the CJA system. The questionable effectiveness of the current system in elevating the international standing of Taiwan's higher-education institutions calls for a re-examination of the FAP system. We recommend a reduction in the relative importance of CJA score in the FAP system, adopting more rigorous metrics such as the h-index for evaluating research quality, and supporting more research aimed at improving the FAP system.

The role of collegiality in academic review, promotion, and tenure

Review, promotion, and tenure (RPT) processes at universities typically assess candidates along three dimensions: research, teaching, and service. In recent years, some have argued for the inclusion of a controversial fourth criterion: collegiality. While collegiality plays a role in the morale and effectiveness of academic departments, it is amorphic and difficult to assess, and could be misused to stifle dissent or enforce homogeneity. Despite this, some institutions have opted to include this additional element in their RPT documents and processes, but it is unknown the extent of this practice and how it varies across institution type and disciplinary units. This study is based on two sets of data: survey data collected as part of a project that explored the publishing decisions of faculty and how these related to perceived importance in RPT processes, and 864 RPT documents collected from 129 universities from the United States and Canada. We analysed these RPT documents to determine the degree to which collegiality and related terms are mentioned, if they are defined, and if and how they may be assessed during the RPT process. Results show that when collegiality and related terms appear in these documents they are most often just briefly mentioned. It is less common for collegiality and related terms to be defined or assessed in RPT documents. Although the terms are mentioned across all types of institutions, there is a statistically significant difference in how prevalent they are at each. Collegiality is more commonly mentioned in the documents of doctoral research-focused universities (60%), than of master’s universities and colleges (31%) or baccalaureate colleges (15%). Results from the accompanying survey of faculty also support this finding: individuals from R-Types were more likely to perceive collegiality to be a factor in their RPT processes. We conclude that collegiality likely plays an important role in RPT processes, whether it is explicitly acknowledged in policies and guidelines or not, and point to several strategies in how it might be best incorporated in the assessment of academic careers.

Education research is still the hardest science: a proposal for improving its trustworthiness and usability

In this essay, we argue that colleges of education, particularly those at research-intensive institutions, favor simplistic notions of scholarly impact and that this trend has concerning implications for the field, for researchers, and for the public at large. After describing the challenges and shortcomings of the current models of research assessment in education, we outline an alternative proposal in which trustworthiness and usability of research would complement traditional metrics of scholarly relevance. This proposal encourages a twofold approach to research assessment that involves (1) a more thorough analysis of the limitations and problems generated by the use of simplistic notions of scholarly impact, and (2) a commitment to the implementation of more equitable systems based on a broader range of assessment measures to assess faculty research contributions.

SDGs: A Responsible Research Assessment Tool toward Impactful Business Research

An alternative research assessment (RA) tool was constructed to assess the relatedness of published business school research to the United Nations’ 17 Sustainable Development Goals (SDGs). The RA tool was created using Leximancer™, an on-line cloud-based text analytic software tool, that identified core themes within the SDG framework. Eight (8) core themes were found to define the ‘spirit of the SDGs’: Sustainable Development, Governance, Vulnerable Populations, Water, Gross Domestic Product (GDP), Food Security, Restoration, and Public Health. These themes were compared to the core themes found in the content of 4576 academic articles published in 2019 in journals that comprise the Financial Times (FT) 50 list. The articles’ relatedness to the SDG themes were assessed. Overall, 10.6% of the themes found in the FT50 journal articles had an explicit relationship to the SDG themes while 24.5% were implied. Themes generated from machine learning (ML), augmented by researcher judgement (to account for synonyms, similar concepts, and discipline specific examples), improved the robustness of the relationships found between the SDG framework and the published articles. Although there are compelling reasons for business schools to focus research on advancing the SDGs, this study and others highlight that there is much opportunity for improvement. Recommendations are made to better align academic research with the SDGs, influencing how business school faculty and their schools prioritize research and its role in the world.

How faculty define quality, prestige, and impact of academic journals

Despite the calls for change, there is significant consensus that when it comes to evaluating publications, review, promotion, and tenure processes should aim to reward research that is of high "quality," is published in "prestigious" journals, and has an "impact." Nevertheless, such terms are highly subjective and present challenges to ascertain precisely what such research looks like. Accordingly, this article responds to the question: how do faculty from universities in the United States and Canada define the terms quality, prestige, and impact of academic journals? We address this question by surveying 338 faculty members from 55 different institutions in the U.S. and Canada. While relying on self-reported definitions that are not linked to their behavior, this study’s findings highlight that faculty often describe these distinct terms in overlapping ways. Additionally, results show that marked variance in definitions across faculty does not correspond to demographic characteristics. This study’s results highlight the subjectivity of common research terms and the importance of implementing evaluation regimes that do not rely on ill-defined concepts and may be context specific.

Understanding the onset of hot streaks across artistic, cultural, and scientific careers

Across a range of creative domains, individual careers are characterized by hot streaks, which are bursts of high-impact works clustered together in close succession. Yet it remains unclear if there are any regularities underlying the beginning of hot streaks. Here, we analyze career histories of artists, film directors, and scientists, and develop deep learning and network science methods to build high-dimensional representations of their creative outputs. We find that across all three domains, individuals tend to explore diverse styles or topics before their hot streak, but become notably more focused after the hot streak begins. Crucially, hot streaks appear to be associated with neither exploration nor exploitation behavior in isolation, but a particular sequence of exploration followed by exploitation, where the transition from exploration to exploitation closely traces the onset of a hot streak. Overall, these results may have implications for identifying and nurturing talents across a wide range of creative domains.

Primate conservation: Lessons learned in the last 20 years can guide future efforts

Twenty years ago, we published an assessment of the threats facing primates and with the passing of two decades, we re-evaluate identified threats, consider emerging pressures, identify exciting new avenues of research, and tackle how to change the system to rapidly advance primate and primate habitat conservation. Habitat destruction and hunting have increased, the danger of looming climate change is clearer, and there are emerging threats such as the sublethal effects of microplastics and pesticides. Despite these negative developments, protected areas are increasing, exciting new tools are now available, and the number of studies has grown exponentially. Many of the changes that need to occur to make rapid progress in primate conservation are in our purview to modify. We identify several dimensions indicating the time is right to make large advances; however, the question that remains is do we have the will to prevent widespread primate annihilation and extinction?

Is the open access citation advantage real? A systematic review of the citation of open access and subscription-based articles

AIMS

Over the last two decades, the existence of an open access citation advantage (OACA)-increased citation of articles made available open access (OA)-has been the topic of much discussion. While there has been substantial research to address this question, findings have been contradictory and inconclusive. We conducted a systematic review to compare studies of citations to OA and non-OA articles.

METHODS

A systematic search of 17 databases attempted to capture all relevant studies authored since 2001. The protocol was registered in Open Science Framework. We included studies with a direct comparison between OA and non-OA items and reported article-level citation as an outcome. Both randomized and non-randomized studies were included. No limitations were placed on study design, language, or publication type.

RESULTS

A total of 5,744 items were retrieved. Ultimately, 134 items were identified for inclusion. 64 studies (47.8%) confirmed the existence of OACA, while 37 (27.6%) found that it did not exist, 32 (23.9%) found OACA only in subsets of their sample, and 1 study (0.8%) was inconclusive. Studies with a focus on multiple disciplines were significantly positively associated with finding that OACA exists in subsets, and are less associated with finding that OACA did not exist. In the critical appraisal of the included studies, 3 were found to have an overall low risk of bias. Of these, one found that an OACA existed, one found that it did not, and one found that an OACA occurred in subsets.

CONCLUSIONS

As seen through the large number of studies identified for this review, OACA is a topic of continuing interest. Quality and heterogeneity of the component studies pose challenges for generalization. The results suggest the need for reporting guidelines for bibliometrics studies.

Preparing for tenure at a research-intensive university

At research-intensive universities in the United States, eligible faculty must generally excel in research, teaching and service in order to receive tenure. To meet these high standards, junior faculty should begin planning for a strong tenure case from their first day on the job. Here, we provide practical information, commentary and advice on how biomedical faculty at research-intensive institutions can prepare strategically for a successful tenure review.

From intent to implementation: Factors affecting public involvement in life science research

Public involvement is key to closing the gap between research production and research use, and the only way to achieving ultimate transparency in science. The majority of life science research is not public-facing, but is funded by the public and impacts communities. We undertook an exploratory survey of researchers within the life sciences to better understand their views and perceived challenges to involving the public in their research. As survey response rate could not be determined, interpretation of the results must be cautious. We had a valid response cohort of n = 110 researchers, of whom 90% were primarily laboratory based. Using a mixed methods approach, we demonstrate that a top-down approach is key to motivate progression of life scientists from feeling positive towards public involvement to actually engaging in it. Researchers who viewed public involvement as beneficial to their research were more likely to have direct experience of doing it. We demonstrate that the systemic flaws in the way life sciences research enterprise is organised, including the promotion system, hyper-competition, and time pressures are major barriers to involving the public in the scientific process. Scientists are also apprehensive of being involuntarily involved in the current politicized climate; misinformation and publicity hype surrounding science nowadays makes them hesitant to share their early and in-progress research. The time required to deliberate study design and relevance, plan and build relationships for sustained involvement, provide and undertake training, and improve communication in the current research environment is often considered nonpragmatic, particularly for early career researchers. In conclusion, a top-down approach involving institutional incentives and infrastructure appears most effective at transitioning researchers from feeling positive towards public involvement to actually implementing it.

Behavioral Reluctance in Adopting Open Access Publishing: Insights From a Goal-Directed Perspective

Despite growing awareness of the benefits of large-scale open access publishing, individual researchers seem reluctant to adopt this behavior, thereby slowing down the evolution toward a new scientific culture. We outline and apply a goal-directed framework of behavior causation to shed light on this type of behavioral reluctance and to organize and suggest possible intervention strategies. The framework explains behavior as the result of a cycle of events starting with the detection of a discrepancy between a goal and a status quo and the selection of behavior to reduce this discrepancy. We list various factors that may hinder this cycle and thus contribute to behavioral reluctance. After that, we highlight potential remedies to address each of the identified barriers. We thereby hope to point out new ways to think about behavioral reluctances in general, and in relation to open access publishing in particular.

Purposeful Evaluation of Scholarship in the Open Science Era

In most of the world’s countries, scholarship evaluation for tenure and promotion continues to rely on conventional criteria of publications in journals of high impact factor and achievements in securing research funds. Continuing to hire and promote scholars based on these criteria exposes universities to risk because students, directly and indirectly through government funds, are the main source of revenues for academic institutions. At the same time, talented young researchers increasingly look for professors renowned for excellence in mentoring doctoral students and early career researchers. Purposeful scholarship evaluation in the open science era needs to include all three areas of scholarly activity: research, teaching and mentoring, and service to society.

Food-Energy-Water Crises in the United States and China: Commonalities and Asynchronous Experiences Support Integration of Global Efforts

Food, energy, and water (FEW) systems have been recognized as an issue of critical global importance. Understanding the mechanisms that govern the FEW nexus is essential to develop solutions and avoid humanitarian crises of displacement, famine, and disease. The U.S. and China are the world's leading economies. Although the two nations are shaped by fundamentally different political and economic systems, they share FEW trajectories in several complementary ways. These realities place the U.S. and China in unique positions to engage in problem definition, dialogue, actions, and transdisciplinary convergence of research to achieve productive solutions addressing FEW challenges. By comparing the nexus and functions of the FEW systems in the two nations, this perspective aims to facilitate collaborative innovations that reduce disciplinary silos, mitigate FEW challenges, and enhance environmental sustainability. The review of experiences and challenges facing the U.S. and China also offers valuable insights for other nations seeking to achieve sustainable development goals.

Rethinking success, integrity, and culture in research (part 1) - a multi-actor qualitative study on success in science

BACKGROUND

Success shapes the lives and careers of scientists. But success in science is difficult to define, let alone to translate in indicators that can be used for assessment. In the past few years, several groups expressed their dissatisfaction with the indicators currently used for assessing researchers. But given the lack of agreement on what should constitute success in science, most propositions remain unanswered. This paper aims to complement our understanding of success in science and to document areas of tension and conflict in research assessments.

METHODS

We conducted semi-structured interviews and focus groups with policy makers, funders, institution leaders, editors or publishers, research integrity office members, research integrity community members, laboratory technicians, researchers, research students, and former-researchers who changed career to inquire on the topics of success, integrity, and responsibilities in science. We used the Flemish biomedical landscape as a baseline to be able to grasp the views of interacting and complementary actors in a system setting.

RESULTS

Given the breadth of our results, we divided our findings in a two-paper series, with the current paper focusing on what defines and determines success in science. Respondents depicted success as a multi-factorial, context-dependent, and mutable construct. Success appeared to be an interaction between characteristics from the researcher (Who), research outputs (What), processes (How), and luck. Interviewees noted that current research assessments overvalued outputs but largely ignored the processes deemed essential for research quality and integrity. Interviewees suggested that science needs a diversity of indicators that are transparent, robust, and valid, and that also allow a balanced and diverse view of success; that assessment of scientists should not blindly depend on metrics but also value human input; and that quality should be valued over quantity.

CONCLUSIONS

The objective of research assessments may be to encourage good researchers, to benefit society, or simply to advance science. Yet we show that current assessments fall short on each of these objectives. Open and transparent inter-actor dialogue is needed to understand what research assessments aim for and how they can best achieve their objective.

STUDY REGISTRATION

osf.io/33v3m.

Open Accessibility in Education Research: Enhancing the Credibility, Equity, Impact, and Efficiency of Research

Openness is a foundational principle in science. Making the tools and products of scientific research openly accessible advances core aims and values of education researchers, such as the credibility, equity, impact, and efficiency of research. The digital revolution has expanded opportunities for providing greater access to research. In this article, we examine three open-science practices-open data and code, open materials, and open access-that education researchers can use to increase accessibility to the tools and products of research in the field. For each open-science practice, we discuss what the practice is and how it works, its primary benefits, some important limitations and challenges, and two thorny issues.

Academic criteria for promotion and tenure in faculties of medicine: a cross-sectional study of the Canadian U15 universities

Background: The objective of this study was to determine the presence of a set of prespecified criteria used to assess scientists for promotion and tenure within faculties of medicine among the U15 Group of Canadian Research Universities.

Methods: Each faculty guideline for assessing promotion and tenure was reviewed and the presence of five traditional (peer-reviewed publications, authorship order, journal impact factor, grant funding, and national/international reputation) and seven nontraditional (citations, data sharing, publishing in open access mediums, accommodating leaves, alternative ways for sharing research, registering research, using reporting guidelines) criteria were collected by two reviewers.

Results: Among the U15 institutions, four of five traditional criteria (80.0%) were present in at least one promotion guideline, whereas only three of seven nontraditional incentives (42.9%) were present in any promotion guidelines. When assessing full professors, there were a median of three traditional criteria listed, versus one nontraditional criterion.

Conclusion: This study demonstrates that faculties of medicine among the U15 Group of Canadian Research Universities base assessments for promotion and tenure on traditional criteria. Some of these metrics may reinforce problematic practices in medical research. These faculties should consider incentivizing criteria that can enhance the quality of medical research.

Millennial researchers in a metric-driven scholarly world: An international study

The study Investigates the attitudes and practices of early career researchers (ECRs) in regard to citation-based metrics and altmetrics, providing the findings in the light of what might be expected of the millennial generation and in the context of what we already know about researchers in today’s ‘culture of counting’ governed scholarly world. The data were gathered by means of an international survey, informed by a preceding, 3-year qualitative study of 120 ECRs from 7 countries, which obtained 1,600 responses. The main conclusions are: 1, citation indicators play a central and multi-purpose role in scholarly communications; 2, altmetrics are not so popular or widely used, but ECRs are waking up to some of their merits, most notably, discovering the extent to which their papers obtain traction and monitoring impact; 3, there is a strong likelihood that ECRs are going to have to grapple with both citation-based metrics and altmetrics, mainly in order to demonstrate research impact; 4, the Chinese are the most metric using nation, largely because of governmental regulations.

The limitations to our understanding of peer review

Peer review is embedded in the core of our knowledge generation systems, perceived as a method for establishing quality or scholarly legitimacy for research, while also often distributing academic prestige and standing on individuals. Despite its critical importance, it curiously remains poorly understood in a number of dimensions. In order to address this, we have analysed peer review to assess where the major gaps in our theoretical and empirical understanding of it lie. We identify core themes including editorial responsibility, the subjectivity and bias of reviewers, the function and quality of peer review, and the social and epistemic implications of peer review. The high-priority gaps are focused around increased accountability and justification in decision-making processes for editors and developing a deeper, empirical understanding of the social impact of peer review. Addressing this at the bare minimum will require the design of a consensus for a minimal set of standards for what constitutes peer review, and the development of a shared data infrastructure to support this. Such a field requires sustained funding and commitment from publishers and research funders, who both have a commitment to uphold the integrity of the published scholarly record. We use this to present a guide for the future of peer review, and the development of a new research discipline based on the study of peer review.

Ten simple rules for innovative dissemination of research

How we communicate research is changing because of new (especially digital) possibilities. This article sets out 10 easy steps researchers can take to disseminate their work in novel and engaging ways, and hence increase the impact of their research on science and society.

Why we publish where we do: Faculty publishing values and their relationship to review, promotion and tenure expectations

Using an online survey of academics at 55 randomly selected institutions across the US and Canada, we explore priorities for publishing decisions and their perceived importance within review, promotion, and tenure (RPT). We find that respondents most value journal readership, while they believe their peers most value prestige and related metrics such as impact factor when submitting their work for publication. Respondents indicated that total number of publications, number of publications per year, and journal name recognition were the most valued factors in RPT. Older and tenured respondents (most likely to serve on RPT committees) were less likely to value journal prestige and metrics for publishing, while untenured respondents were more likely to value these factors. These results suggest disconnects between what academics value versus what they think their peers value, and between the importance of journal prestige and metrics for tenured versus untenured faculty in publishing and RPT perceptions.

Scientists' incentives and attitudes toward public communication

In an era of large-scale science-related challenges and rapid advancements in groundbreaking science with major societal implications, communicating about science is critical. The profile of science communication has increased over the last few decades, with multiple sectors calling for such activities. As scientists respond to calls for public-facing communication, we need to evaluate where the scientific community stands. We conducted a unique census of science faculty at land-grant universities across the United States intended to spur the next generation of science communicators and research. Despite scientists' strong approval of science communication efforts, potential areas of tension, attributable to lack of institutional support and confidence in communication skills, constrain these efforts.

Use of the Journal Impact Factor in academic review, promotion, and tenure evaluations

We analyzed how often and in what ways the Journal Impact Factor (JIF) is currently used in review, promotion, and tenure (RPT) documents of a representative sample of universities from the United States and Canada. 40% of research-intensive institutions and 18% of master’s institutions mentioned the JIF, or closely related terms. Of the institutions that mentioned the JIF, 87% supported its use in at least one of their RPT documents, 13% expressed caution about its use, and none heavily criticized it or prohibited its use. Furthermore, 63% of institutions that mentioned the JIF associated the metric with quality, 40% with impact, importance, or significance, and 20% with prestige, reputation, or status. We conclude that use of the JIF is encouraged in RPT evaluations, especially at research-intensive universities, and that there is work to be done to avoid the potential misuse of metrics like the JIF.

The complex nature of research dissemination practices among public health faculty researchers

Objective

This study explores the variety of information formats used and audiences targeted by public health faculty in the process of disseminating research.

Methods

The authors conducted semi-structured interviews with twelve faculty members in the School of Public Health at the University of Illinois at Chicago, asking them about their research practices, habits, and preferences.

Results

Faculty scholars disseminate their research findings in a variety of formats intended for multiple audiences, including not only their peers in academia, but also public health practitioners, policymakers, government and other agencies, and community partners.

Conclusion

Librarians who serve public health faculty should bear in mind the diversity of faculty's information needs when designing and improving library services and resources, particularly those related to research dissemination and knowledge translation. Promising areas for growth in health sciences libraries include supporting data visualization, measuring the impact of non-scholarly publications, and promoting institutional repositories for dissemination of research.

Open laboratory notebooks: good for science, good for society, good for scientists

The fundamental goal of the growing open science movement is to increase the efficiency of the global scientific community and accelerate progress and discoveries for the common good. Central to this principle is the rapid disclosure of research outputs in open-access peer-reviewed journals and on pre-print servers. The next bold step in this direction is open laboratory notebooks, where research scientists share their research - including detailed protocols, negative and positive results - online and in near-real-time to synergize with their peers. Here, we highlight the benefits of open lab notebooks to science, society and scientists, and discuss the challenges that this nascent movement is facing. We also present the implementation and progress of our own initiative at openlabnotebooks.org, with more than 20 active contributors after one year of operation.