Data sharing and reuse practices: disciplinary differences and improvements needed

Purpose

This study investigates differences and commonalities in data production, sharing and reuse across the widest range of disciplines yet and identifies types of improvements needed to promote data sharing and reuse.

Design/methodology/approach

The first authors of randomly selected publications from 2018 to 2019 in 20 Scopus disciplines were surveyed for their beliefs and experiences about data sharing and reuse.

Findings

From the 3,257 survey responses, data sharing and reuse are still increasing but not ubiquitous in any subject area and are more common among experienced researchers. Researchers with previous data reuse experience were more likely to share data than others. Types of data produced and systematic online data sharing varied substantially between subject areas. Although the use of institutional and journal-supported repositories for sharing data is increasing, personal websites are still frequently used. Combining multiple existing datasets to answer new research questions was the most common use. Proper documentation, openness and information on the usability of data continue to be important when searching for existing datasets. However, researchers in most disciplines struggled to find datasets to reuse. Researchers' feedback suggested 23 recommendations to promote data sharing and reuse, including improved data access and usability, formal data citations, new search features and cultural and policy-related disciplinary changes to increase awareness and acceptance.

Originality/value

This study is the first to explore data sharing and reuse practices across the full range of academic discipline types. It expands and updates previous data sharing surveys and suggests new areas of improvement in terms of policy, guidance and training programs.

Peer review

The peer review history for this article is available at: https://publons.com/publon/10.1108/OIR-08-2021-0423.

Terms in journal articles associating with high quality: can qualitative research be world-leading?

Purpose

Scholars often aim to conduct high quality research and their success is judged primarily by peer reviewers. Research quality is difficult for either group to identify, however and misunderstandings can reduce the efficiency of the scientific enterprise. In response, we use a novel term association strategy to seek quantitative evidence of aspects of research that are associated with high or low quality.

Design/methodology/approach

We extracted the words and 2–5-word phrases most strongly associated with different quality scores in each of 34 Units of Assessment (UoAs) in the Research Excellence Framework (REF) 2021. We extracted the terms from 122,331 journal articles 2014–2020 with individual REF2021 quality scores.

Findings

The terms associating with high- or low-quality scores vary between fields but relate to writing styles, methods and topics. We show that the first-person writing style strongly associates with higher quality research in many areas because it is the norm for a set of large prestigious journals. We found methods and topics that associate with both high- and low-quality scores. Worryingly, terms associated with educational and qualitative research attract lower quality scores in multiple areas. REF experts may rarely give high scores to qualitative or educational research because the authors tend to be less competent, because it is harder to do world leading research with these themes, or because they do not value them.

Originality/value

This is the first investigation of journal article terms associating with research quality.

¿Qué hace que un tuit sobre un libro sea popular? Análisis de los contenidos más retuiteados creados por editoriales de libros españolas y extranjeras

El objetivo de este artículo es identificar características relacionadas con el contenido de los mensajes más retuiteados creados por editoriales de libros españolas y extranjeras en Twitter. Se ha realizado un análisis de contenido para identificar el tema de los tuits y si incluyen hashtag para el título del libro, imágenes e hipervínculos, y en caso de incluirse, sobre qué son las imágenes y hacia dónde apuntan los enlaces. Como complemento, se ha realizado un análisis de asociación de palabras para identificar qué términos son asociados con cada una de las diferentes editoriales. En general, las editoriales tienden a tuitear sobre ellas mismas y sus libros con fines de marketing. Aproximadamente la mitad de las editoriales tienen cuentas en Twitter. Los tuits más populares de las editoriales españolas suelen contener citas literarias, mientras que los tuits más populares de las editoriales extranjeras tienden más a incluir sorteos. Los editores que buscan comprometerse con lectores potenciales en Twitter podrían considerar las citas y los sorteos para construir su audiencia, además de etiquetar al nombre de usuario del autor (@nombredeusuario) en tuits relacionados con libros para ayudar al autor con su red social.

The high scholarly value of grey literature before and during Covid-19

New academic knowledge in journal articles is partly built on peer reviewed research already published in journals or books. Academics can also draw from non-academic sources, such as the websites of organisations that publish credible information. This article investigates trends in the academic citing of this type of grey literature for 17 health, media, statistics, and large international organisations, with a focus on Covid-19. The results show substantial and steadily increasing numbers of citations to all 17 sites, with larger increases from 2019 to 2020. In 2020, Covid-19 citations to these websites were particularly common for news organisations, the WHO, and the UK Office for National Statistics, apparently for up-to-date information in the rapidly changing circumstances of the pandemic. Except for the UN, the most cited URLs of each organisation were not traditional report-like grey literature but were other types, such as news stories, data, statistics, and general guidance. The Covid-19 citations to most of these websites originated primarily from medical research, commonly for coronavirus data and statistics. Other fields extensively cited some of the non-health websites, as illustrated by social science (including psychology) studies often citing UNESCO. The results confirm that grey literature from major websites has become even more important within academia during the pandemic, providing up-to-date information from credible sources despite a lack of academic peer review. Researchers, reviewers, and editors should accept that it is reasonable to cite this information, when relevant, and evaluators should value academic work that supports these non-academic outputs.

Academic LGBTQ+ Terminology 1900-2021: Increasing Variety, Increasing Inclusivity?

LGBTQ+ labels and terminology in society embed ideological assumptions and affect who gains community support and protection. In academia, terminology is also needed to help define study objects, methods, and goals. Academics therefore need to choose their words to be both precise and appropriate, adjusting to changes in societal language. This article assesses the evolution of LGBTQ+ terminology in the titles and abstracts of academic journal articles since 1900 to identify the main trends. Based on a search of 74 LGBTQ+ terms in Scopus, LGBTQ+ related journal articles have almost continually increased in prevalence since 1900. In parallel, the concept of homosexuality that dominated early research has almost disappeared, being replaced by the word gay or more specific terms, such as lesbian or bisexual. Transexual terminology has also been supplanted by transgender and trans* terminology. At various points in time other LGBTQ+ terms have emerged with activist, health professional and academic origins. These include multiple acronyms, inclusive phrases, and activity-specific phrases (e.g., men who have sex with men) that are not used by the LGBTQ+ community. Currently, no terminologies are dominant, with this plurality probably reflecting differing research needs.

Research Co-authorship 1900–2020: Continuous, universal, and ongoing expansion

Research co-authorship is useful to combine different skillsets, especially for applied problems. Whilst it has increased over the last century, it is unclear whether this increase is universal across academic fields and which fields co-author the most and least. In response, this article assesses changes in the rate of journal article co-authorship 1900–2020 for all 27 Scopus broad fields and all 332 Scopus narrow fields. Whilst all broad fields have experienced reasonably continuous growth in co-authorship, in 2020 there were substantial disciplinary differences, from Arts and Humanities (1.3 authors) to Immunology and Microbiology (6 authors). All 332 Scopus narrow fields also experienced an increase in the average number of authors. Immunology and Classics are extreme Scopus narrow fields, as exemplified by 9.6 authors per Journal for ImmunoTherapy of Cancer article, whilst 93% of Trends in Classics articles were solo in 2020. The reason for this large difference seems to be the need for multiple complementary methods in Immunology, making it fundamentally a team science. Finally, the reasonably steady and universal increases in academic coauthorship over 121 years show no sign of slowing, suggesting that ever expanding teams are a central part of current professional science.

Journal and disciplinary variations in academic open peer review anonymity, outcomes, and length

Understanding more about variations in peer review is essential to ensure that editors and reviewers harness it effectively in existing and new formats, including for mega-journals and when published online. This article analyzes open reviews from the MDPI suite of journals to identify commonalities and differences from a simplistic quantitative perspective, focusing on reviewer anonymity, review length, and review outcomes. The sample contained 45,385 first round open reviews from published standard journal articles in 288 MDPI journals classified into one or more Scopus disciplinary areas (Health Sciences; Life Sciences; Physical Sciences; Social Sciences). The eight main findings include substantial differences between journals and disciplines in review lengths, reviewer anonymity, review outcomes, and the use of attachments. In particular, Physical Sciences journal reviews tended to be stricter and were more likely to be anonymous. Life Sciences and Social Sciences reviews were the longest overall. Signed reviews tend to be 15% longer (perhaps to be more careful or polite) but gave similar decisions to anonymous reviews. Finally, reviews with major revision outcomes tended to be 68% longer than reviews with for minor revision outcomes, except in a few journals. In conclusion, signing reviews does not seem to threaten the validity of peer review outcomes and authors, editors, and reviewers of multidisciplinary articles should be aware of substantial field differences in what constitutes an appropriate review.

Can the quality of published academic journal articles be assessed with machine learning?

Formal assessments of the quality of the research produced by departments and universities are now conducted by many countries to monitor achievements and allocate performance-related funding. These evaluations are hugely time consuming if conducted by post-publication peer review and are simplistic if based on citations or journal impact factors. This article investigates whether machine learning could help reduce the burden of peer review by using citations and metadata to learn how to score articles from a sample assessed by peer review. An experiment is used to underpin the discussion, attempting to predict journal citation thirds, as a proxy for article quality scores, for all Scopus narrow fields from 2014 to 2020. The results show that these proxy quality thirds can be predicted with above baseline accuracy in all 326 narrow fields, with Gradient Boosting Classifier, Random Forest Classifier, or Multinomial Naïve Bayes being the most accurate in nearly all cases. Nevertheless, the results partly leverage journal writing styles and topics, which are unwanted for some practical applications and cause substantial shifts in average scores between countries and between institutions within a country. There may be scope for predicting articles scores when the predictions have the highest probability.

Peer Review

https://publons.com/publon/10.1162/qss_a_00185

Small female citation advantages for US journal articles in medicine

Female under-representation continues in senior roles within academic medicine, potentially influenced by a perception that female research has less citation impact. This article provides systematic evidence of (a) female participation rates from the perspective of published journal articles in 46 Scopus medical subject categories 1996–2018 and (b) gender differences in citation rates 1996–2014. The results show female proportion increases 1996–2018 in all fields and a female majority of first-authored articles in two-fifths of categories, but substantial differences between fields. A paper is 7.3 times more likely to have a female first author in Obstetrics and Gynaecology than in Orthopaedics and Sports Medicine. Only three fields had a female last author majority by 2018, a probable side effect of ongoing problems with appointing female leaders. Female first-authored research tended to be more cited than male first-authored research in most fields (59%), although with a maximum difference of only 5.1% (log-transformed normalised citations). In contrast, male last-authored research tends to be more cited than female last-authored research, perhaps due to cases where a senior male has attracted substantial funding for a project. These differences increase if team sizes are not accounted for in the calculations. Since female first-authored research is cited slightly more than male first-authored research, properly analysed bibliometric data considering career gaps should not disadvantage female candidates for senior roles.

Covid-19 refereeing duration and impact in major medical journals

Two partly conflicting academic pressures from the seriousness of the Covid-19 pandemic are the need for faster peer review of Covid-19 health-related research and greater scrutiny of its findings. This paper investigates whether decreases in peer review durations for Covid-19 articles were universal across 97 major medical journals, Nature, Science, and Cell. The results suggest that on average, Covid-19 articles submitted during 2020 were reviewed 1.7–2.1 times faster than non-Covid-19 articles submitted during 2017–2020. Nevertheless, whilst the review speed of Covid-19 research was particularly fast during the first five months (1.9–3.4 times faster) of the pandemic (January–May 2020), this speed advantage was no longer evident for articles submitted November–December 2020. Faster peer review also associates with higher citation impact for Covid-19 articles in the same journals, suggesting it did not usually compromise the scholarly impact of important Covid-19 research. Overall, then, it seems that core medical and general journals responded quickly but carefully to the pandemic, although the situation returned closer to normal within a year.

Peer Review

https://publons.com/publon/10.1162/qss_a_00176

Scopus 1900–2020: Growth in articles, abstracts, countries, fields, and journals

Scientometric research often relies on large-scale bibliometric databases of academic journal articles. Long term and longitudinal research can be affected if the composition of a database varies over time, and text processing research can be affected if the percentage of articles with abstracts changes. This article therefore assesses changes in the magnitude of the coverage of a major citation index, Scopus, over 121 years from 1900. The results show sustained exponential growth from 1900, except for dips during both world wars, and with increased growth after 2004. Over the same period, the percentage of articles with 500+ character abstracts increased from 1% to 95%. The number of different journals in Scopus also increased exponentially, but slowing down from 2010, with the number of articles per journal being approximately constant until 1980, then tripling due to megajournals and online-only publishing. The breadth of Scopus, in terms of the number of narrow fields with substantial numbers of articles, simultaneously increased from one field having 1000 articles in 1945 to 308 in 2020. Scopus’s international character also radically changed from 68% of first authors from Germany and the USA in 1900 to just 17% in 2020, with China dominating (25%).

Peer Review

https://publons.com/publon/10.1162/qss_a_00177

Researching women and men 1996–2020: Is androcentrism still dominant?

This article assesses the balance of research concerning women and men over the past quarter century using the crude heuristic of counting Scopus-indexed journal articles relating to women or men, as suggested by their titles or abstracts. A manual checking procedure together with a word-based heuristic was used to identify whether an article related to women or men. The heuristic includes both explicit mentions of women and men, implicit mentions, and a set of gender-focused health issues and medical terminology. Based on the results, more published articles now relate to women than to men. Moreover, more than twice as many articles relate exclusively to women than exclusively to men, with the ratio increasing from 2.16 to 1 in 1996 to 2.25 to 1 in 2020. Monogender articles mostly addressed primarily female health issues (maternity, breast cancer, cervical cancer) with fewer about primarily male health issues (testicular cancer, pancreatic cancer, health needs of men who have sex with men). Some articles also explicitly addressed gender inequality, such as empowering women entrepreneurs. The findings suggest that the androcentrism of early science has eroded in terms of research topics. This apparent progress should be encouraging for women researchers and society.

Which types of online evidence show the nonacademic benefits of research? Websites cited in UK impact case studies

While funders increasingly request evidence of the societal benefits of research, all academics in the UK must periodically provide this information to gain part of their block funding within the Research Excellence Framework (REF). The impact case studies produced in the UK are public and can therefore be used to gain insights into the types of sources used to justify societal impact claims. This study focuses on the URLs cited as evidence in the last public REF to help researchers and resource providers to understand what types can be used and the disciplinary differences in their uptake. Based on a new semiautomatic method to classify the URLs cited in impact case studies, the results show that there are a few key online types of source for most broad fields, but these sources differ substantially between subject areas. For example, news websites are more important in some fields than others, and YouTube is sometimes used for multimedia evidence in the arts and humanities. Knowledge of the common sources selected independently by thousands of researchers may help others to identify suitable sources for the complex task of evidencing societal impacts.

A Bayesian hurdle quantile regression model for citation analysis with mass points at lower values

Quantile regression presents a complete picture of the effects on the location, scale, and shape of the dependent variable at all points, not just the mean. We focus on two challenges for citation count analysis by quantile regression: discontinuity and substantial mass points at lower counts. A Bayesian hurdle quantile regression model for count data with a substantial mass point at zero was proposed by King and Song (2019). It uses quantile regression for modeling the nonzero data and logistic regression for modeling the probability of zeros versus nonzeros. We show that substantial mass points for low citation counts will almost certainly also affect parameter estimation in the quantile regression part of the model, similar to a mass point at zero. We update the King and Song model by shifting the hurdle point past the main mass points. This model delivers more accurate quantile regression for moderately to highly cited articles, especially at quantiles corresponding to values just beyond the mass points, and enables estimates of the extent to which factors influence the chances that an article will be low cited. To illustrate the potential of this method, it is applied to simulated citation counts and data from Scopus.

Exploring nurses' online perspectives and social networks during a global pandemic COVID-19

Objectives

Examine the online interactions, social networks, and perspectives of nursing actors on COVID‐19 from conversations on Twitter to understand how the profession responded to this global pandemic.

Design

Mixed methods.

Sample

Ten‐thousand five‐hundred and seventy‐four tweets by 2790 individuals and organizations.

Measurements

NodeXL software was used for social network analysis to produce a network visualization. The betweenness centrality algorithm identified key users who were influential in COVID‐19 related conversations on Twitter. Inductive content analysis enabled exploration of tweet content. A communicative figurations framework guided the study.

Results

Nursing actors formed different social groupings, and communicated with one another across groups. Tweets covered four themes; (1) outbreak and clinical management of the infectious disease, (2) education and information sharing, (3) social, economic, and political context, and (4) working together and supporting each other.

Conclusion

In addition to spreading knowledge, nurses tried to reach out through social media to political and healthcare leaders to advocate for improvements needed to address COVID‐19. However, they primarily conversed within their own professional community. Action is needed to better understand how social media is and can be used by nurses for health communication, and to improve their preparedness to be influential on social media beyond the nursing community.

Social media users produce more affect that supports cultural values, but are more influenced by affect that violates cultural values

Although social media plays an increasingly important role in communication around the world, social media research has primarily focused on Western users. Thus, little is known about how cultural values shape social media behavior. To examine how cultural affective values might influence social media use, we developed a new sentiment analysis tool that allowed us to compare the affective content of Twitter posts in the United States (55,867 tweets, 1,888 users) and Japan (63,863 tweets, 1,825 users). Consistent with their respective cultural affective values, U.S. users primarily produced positive (vs. negative) posts, whereas Japanese users primarily produced low (vs. high) arousal posts. Contrary to cultural affective values, however, U.S. users were more influenced by changes in others' high arousal negative (e.g., angry) posts, whereas Japanese were more influenced by changes in others' high arousal positive (e.g., excited) posts. These patterns held after controlling for differences in baseline exposure to affective content, and across different topics. Together, these results suggest that across cultures, while social media users primarily produce content that supports their affective values, they are more influenced by content that violates those values. These findings have implications for theories about which affective content spreads on social media, and for applications related to the optimal design and use of social media platforms around the world. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Alternative medicines worth researching? Citation analyses of acupuncture, chiropractic, homeopathy, and osteopathy 1996-2017

Some complementary and alternative medicines (CAM) are frequently criticised for being based on faith rather than scientific evidence. Despite this, researchers, academic departments, and institutes teach and investigate them. This article assesses whether the scholarship produced by four CAMs is valued by the academic community in terms of citations, and whether the level of citations received might be detrimental to academic authors' careers. Based on an analysis of acupuncture, chiropractic, homeopathy, and osteopathy journal articles indexed in Scopus 1996-2020, the results show that the prevalence of the four areas vary substantially internationally, with acupuncture eclipsing the others in East Asia but homeopathy being more common in India and Brazil. The main broad fields publishing these specialties are Medicine, Nursing, Health Professions, Veterinary Science, and Neuroscience. Whilst the research tends to be cited at a below average rate in most broad fields (n = 27) and years (1996-2017), acupuncture, chiropractic, and homeopathy are exceptions in some broad fields, including some core areas. Thus, studying these alternative medicines may not always lead to research that tends to be ignored in academia, even if many scientists disparage it. As a corollary, citation analysis cannot be relied on to give low scores to widely disparaged areas of scholarship.

Are data repositories fettered? A survey of current practices, challenges and future technologies

Purpose

The purpose of this study is to explore current practices, challenges and technological needs of different data repositories.

Design/methodology/approach

An online survey was designed for data repository managers, and contact information from the re3data, a data repository registry, was collected to disseminate the survey.

Findings

In total, 189 responses were received, including 47% discipline specific and 34% institutional data repositories. A total of 71% of the repositories reporting their software used bespoke technical frameworks, with DSpace, EPrint and Dataverse being commonly used by institutional repositories. Of repository managers, 32% reported tracking secondary data reuse while 50% would like to. Among data reuse metrics, citation counts were considered extremely important by the majority, followed by links to the data from other websites and download counts. Despite their perceived usefulness, repository managers struggle to track dataset citations. Most repository managers support dataset and metadata quality checks via librarians, subject specialists or information professionals. A lack of engagement from users and a lack of human resources are the top two challenges, and outreach is the most common motivator mentioned by repositories across all groups. Ensuring findable, accessible, interoperable and reusable (FAIR) data (49%), providing user support for research (36%) and developing best practices (29%) are the top three priorities for repository managers. The main recommendations for future repository systems are as follows: integration and interoperability between data and systems (30%), better research data management (RDM) tools (19%), tools that allow computation without downloading datasets (16%) and automated systems (16%).

Originality/value

This study identifies the current challenges and needs for improving data repository functionalities and user experiences.

Peer review

The peer review history for this article is available at: https://publons.com/publon/10.1108/OIR-04-2021-0204

Domestic researchers with longer careers generate higher average citation impact but it does not increase over time

Information about the relative strengths of scholars is needed for the efficient running of knowledge systems. Because academic research requires many skills, more experienced researchers might produce better research and attract more citations. This article assesses career citation impact changes 2001–2016 for domestic researchers (definition: first and last Scopus journal article in the same country) from the 12 nations with most Scopus documents. Careers are analyzed longitudinally, so that changes are not due to personnel evolution, such as researchers leaving or entering a country. The results show that long-term domestic researchers do not tend to improve their citation impact over time but tend to achieve their average citation impact by their first or second Scopus journal article. In some countries, this citation impact subsequently declines. These longer-term domestic researchers have higher citation impact than the national average in all countries, however, whereas scholars publishing only one journal article have substantially lower citation impact in all countries. The results are consistent with an efficiently functioning researcher selection system but cast slight doubt on the long-term citation impact potential of long-term domestic researchers. Research and funding policies may need to accommodate these patterns when citation impact is a relevant indicator.

Mental Health Discourses on Twitter during Mental Health Awareness Week

Promoting health-related campaigns on Twitter has increasingly become a world-wide choice to raise awareness and disseminate health information. Data retrieved from Twitter are now being used to explore how users express their views, attitudes and personal experiences of health-related issues. We focused on Twitter discourse reproduced during Mental Health Awareness Week 2017 by examining 1,200 tweets containing the keywords 'mental health', 'mental illness', 'mental disorders' and '#MHAW'. The analysis revealed 'awareness and advocacy', 'stigmatization', and 'personal experience of mental health/illness' as the central discourses within the sample. The article concludes with some recommendations for future research on digitally-mediated health communication.

Researchers' attitudes towards the h-index on Twitter 2007-2020: criticism and acceptance

The h-index is an indicator of the scientific impact of an academic publishing career. Its hybrid publishing/citation nature and inherent bias against younger researchers, women, people in low resourced countries, and those not prioritizing publishing arguably give it little value for most formal and informal research evaluations. Nevertheless, it is well-known by academics, used in some promotion decisions, and is prominent in bibliometric databases, such as Google Scholar. In the context of this apparent conflict, it is important to understand researchers’ attitudes towards the h-index. This article used public tweets in English to analyse how scholars discuss the h-index in public: is it mentioned, are tweets about it positive or negative, and has interest decreased since its shortcomings were exposed? The January 2021 Twitter Academic Research initiative was harnessed to download all English tweets mentioning the h-index from the 2006 start of Twitter until the end of 2020. The results showed a constantly increasing number of tweets. Whilst the most popular tweets unapologetically used the h-index as an indicator of research performance, 28.5% of tweets were critical of its simplistic nature and others joked about it (8%). The results suggest that interest in the h-index is still increasing online despite scientists willing to evaluate the h-index in public tending to be critical. Nevertheless, in limited situations it may be effective at succinctly conveying the message that a researcher has had a successful publishing career.

Pot, kettle: Nonliteral titles aren’t (natural) science

Researchers may be tempted to attract attention through poetic titles for their publications, but would this be mistaken in some fields? Although poetic titles are known to be common in medicine, it is not clear whether the practice is widespread elsewhere. This article investigates the prevalence of poetic expressions in journal article titles from 1996–2019 in 3.3 million articles from all 27 Scopus broad fields. Expressions were identified by manually checking all phrases with at least five words that occurred at least 25 times, finding 149 stock phrases, idioms, sayings, literary allusions, film names, and song titles or lyrics. The expressions found are most common in the social sciences and the humanities. They are also relatively common in medicine, but almost absent from engineering and the natural and formal sciences. The differences may reflect the less hierarchical and more varied nature of the social sciences and humanities, where interesting titles may attract an audience. In engineering, natural science, and formal science fields, authors should take extra care with poetic expressions in case their choice is judged inappropriate. This includes interdisciplinary research overlapping these areas. Conversely, reviewers of interdisciplinary research involving the social sciences should be more tolerant of poetic license.

Coronavirus research before 2020 is more relevant than ever, especially when interpreted for COVID-19

The speed with which biomedical specialists were able to identify and characterize COVID-19 was partly due to prior research with other coronaviruses. Early epidemiological comparisons with Severe Acute Respiratory Syndrome (SARS) and Middle East Respiratory Syndrome (MERS), also made it easier to predict COVID-19’s likely spread and lethality. This article assesses whether academic interest in prior coronavirus research has translated into interest in the primary source material, using Mendeley reader counts for early academic impact evidence. The results confirm that SARS and MERS research in 2008–2017 experienced anomalously high increases in Mendeley readers in April–May 2020. Nevertheless, studies learning COVID-19 lessons from SARS and MERS or using them as a benchmark for COVID-19 have generated much more academic interest than primary studies of SARS or MERS. Thus, research that interprets prior relevant research for new diseases when they are discovered seems to be particularly important to help researchers to understand its implications in the new context.

A thematic analysis of highly retweeted early COVID-19 tweets: consensus, information, dissent and lockdown life

Purpose

Public attitudes towards COVID-19 and social distancing are critical in reducing its spread. It is therefore important to understand public reactions and information dissemination in all major forms, including on social media. This article investigates important issues reflected on Twitter in the early stages of the public reaction to COVID-19.

Design/methodology/approach

A thematic analysis of the most retweeted English-language tweets mentioning COVID-19 during March 10–29, 2020.

Findings

The main themes identified for the 87 qualifying tweets accounting for 14 million retweets were: lockdown life; attitude towards social restrictions; politics; safety messages; people with COVID-19; support for key workers; work; and COVID-19 facts/news.

Research limitations/implications

Twitter played many positive roles, mainly through unofficial tweets. Users shared social distancing information, helped build support for social distancing, criticised government responses, expressed support for key workers and helped each other cope with social isolation. A few popular tweets not supporting social distancing show that government messages sometimes failed.

Practical implications

Public health campaigns in future may consider encouraging grass roots social web activity to support campaign goals. At a methodological level, analysing retweet counts emphasised politics and ignored practical implementation issues.

Originality/value

This is the first qualitative analysis of general COVID-19-related retweeting.

COVID-19 publications: Database coverage, citations, readers, tweets, news, Facebook walls, Reddit posts

The COVID-19 pandemic requires a fast response from researchers to help address biological, medical, and public health issues to minimize its impact. In this rapidly evolving context, scholars, professionals, and the public may need to identify important new studies quickly. In response, this paper assesses the coverage of scholarly databases and impact indicators during March 21, 2020 to April 18, 2020. The rapidly increasing volume of research is particularly accessible through Dimensions, and less through Scopus, the Web of Science, and PubMed. Google Scholar’s results included many false matches. A few COVID-19 papers from the 21,395 in Dimensions were already highly cited, with substantial news and social media attention. For this topic, in contrast to previous studies, there seems to be a high degree of convergence between articles shared in the social web and citation counts, at least in the short term. In particular, articles that are extensively tweeted on the day first indexed are likely to be highly read and relatively highly cited 3 weeks later. Researchers needing wide scope literature searches (rather than health-focused PubMed or medRxiv searches) should start with Dimensions (or Google Scholar) and can use tweet and Mendeley reader counts as indicators of likely importance.

A gender equality paradox in academic publishing: Countries with a higher proportion of female first-authored journal articles have larger first-author gender disparities between fields

Current attempts to address the shortfall of female researchers in science, technology, engineering, and mathematics (STEM) have not yet succeeded, despite other academic subjects having female majorities. This article investigates the extent to which gender disparities are subject-wide or nation-specific by a first-author gender comparison of 30 million articles from all 27 Scopus broad fields within the 31 countries with the most Scopus-indexed articles 2014–2018. The results show overall and geocultural patterns as well as individual national differences. Almost half of the subjects were always more male (seven; e.g., Mathematics) or always more female (six; e.g., Immunology & Microbiology) than the national average. A strong overall trend (Spearman correlation 0.546) is for countries with a higher proportion of female first-authored research to also have larger differences in gender disparities between fields (correlation 0.314 for gender ratios). This confirms the international gender equality paradox previously found for degree subject choices: Increased gender equality overall associates with moderately greater gender differentiation between subjects. This is consistent with previous United States-based claims that gender differences in academic careers are partly due to (socially constrained) gender differences in personal preferences. Radical solutions may therefore be needed for some STEM subjects to overcome gender disparities.

Greater female first author citation advantages do not associate with reduced or reducing gender disparities in academia

Ongoing problems attracting women into many Science, Technology, Engineering and Mathematics (STEM) subjects have many potential explanations. This article investigates whether the possible undercitation of women associates with lower proportions of, or increases in, women in a subject. It uses six million articles published in 1996–2012 across up to 331 fields in six mainly English-speaking countries: Australia, Canada, Ireland, New Zealand, the United Kingdom and the United States. The proportion of female first- and last-authored articles in each year was calculated and 4,968 regressions were run to detect first-author gender advantages in field normalized article citations. The proportion of female first authors in each field correlated highly between countries and the female first-author citation advantages derived from the regressions correlated moderately to strongly between countries, so both are relatively field specific. There was a weak tendency in the United States and New Zealand for female citation advantages to be stronger in fields with fewer women, after excluding small fields, but there was no other association evidence. There was no evidence of female citation advantages or disadvantages to be a cause or effect of changes in the proportions of women in a field for any country. Inappropriate uses of career-level citations are a likelier source of gender inequities.

All downhill from the PhD? The typical impact trajectory of U.S. academic careers

Within academia, mature researchers tend to be more senior, but do they also tend to write higher impact articles? This article assesses long-term publishing (16+ years) United States (U.S.) researchers, contrasting them with shorter-term publishing researchers (1, 6, or 10 years). A long-term U.S. researcher is operationalized as having a first Scopus-indexed journal article in exactly 2001 and one in 2016–2019, with U.S. main affiliations in their first and last articles. Researchers publishing in large teams (11+ authors) were excluded. The average field and year normalized citation impact of long- and shorter-term U.S. researchers’ journal articles decreases over time relative to the national average, with especially large falls for the last articles published, which may be at least partly due to a decline in self-citations. In many cases researchers start by publishing above U.S. average citation impact research and end by publishing below U.S. average citation impact research. Thus, research managers should not assume that senior researchers will usually write the highest impact papers.

Does the use of open, non-anonymous peer review in scholarly publishing introduce bias? Evidence from the F1000Research post-publication open peer review publishing model

As part of moves towards open knowledge practices, making peer review open is cited as a way to enable fuller scrutiny and transparency of assessments around research. There are now many flavours of open peer review in use across scholarly publishing, including where reviews are fully attributable and the reviewer is named. This study examines whether there is any evidence of bias in two areas of common critique of open, non-anonymous (named) peer review – and used in the post-publication, peer review system operated by the open-access scholarly publishing platform F1000Research. First, is there evidence of potential bias where a reviewer based in a specific country assesses the work of an author also based in the same country? Second, are reviewers influenced by being able to see the comments and know the origins of a previous reviewer? Based on over 4 years of open peer review data, we found some weak evidence that being based in the same country as an author may influence a reviewer’s decision, while there was insufficient evidence to conclude that being able to read an existing published review prior to submitting a review encourages conformity. Thus, while immediate publishing of peer review reports appears to be unproblematic, caution may be needed when selecting same-country reviewers in open systems if other studies confirm these results.

How does nursing research differ internationally? A bibliometric analysis of six countries

BACKGROUND

International nursing research comparisons can give a new perspective on a nation's output by identifying strengths and weaknesses.

AIM

This article compares strengths in nursing research between six mainly English-speaking nations (Australia, Canada, Ireland, New Zealand, United Kingdom and United States).

METHODS

Journal authorship (percentage of first authorship by nationality) and article keywords were compared for Scopus-indexed journal articles 2008-2018. Three natural language processing strategies were assessed for identifying statistically significant international differences in the use of keywords or phrases.

RESULTS

Journal author nationality was not a good indicator of international differences in research specialisms, but keyword and phrase differences were more promising especially if both are used. For this, the part of speech tagging and lemmatisation text processing strategies were helpful but not named entity recognition. The results highlight aspects of nursing research that were absent in some countries, such as papers about nursing administration and management.

CONCLUSION

Researchers outside the United States should consider the importance of researching specific patient groups, diseases, treatments, skills, research methods and social perspectives for unresearched gaps with national relevance. From a methods perspective, keyword and phrase differences are useful to reveal international differences in nursing research topics.

Automatically detecting open academic review praise and criticism

Purpose

Peer reviewer evaluations of academic papers are known to be variable in content and overall judgements but are important academic publishing safeguards. This article introduces a sentiment analysis program, PeerJudge, to detect praise and criticism in peer evaluations. It is designed to support editorial management decisions and reviewers in the scholarly publishing process and for grant funding decision workflows. The initial version of PeerJudge is tailored for reviews from F1000Research's open peer review publishing platform.

Design/methodology/approach

PeerJudge uses a lexical sentiment analysis approach with a human-coded initial sentiment lexicon and machine learning adjustments and additions. It was built with an F1000Research development corpus and evaluated on a different F1000Research test corpus using reviewer ratings.

Findings

PeerJudge can predict F1000Research judgements from negative evaluations in reviewers' comments more accurately than baseline approaches, although not from positive reviewer comments, which seem to be largely unrelated to reviewer decisions. Within the F1000Research mode of post-publication peer review, the absence of any detected negative comments is a reliable indicator that an article will be ‘approved’, but the presence of moderately negative comments could lead to either an approved or approved with reservations decision.

Originality/value

PeerJudge is the first transparent AI approach to peer review sentiment detection. It may be used to identify anomalous reviews with text potentially not matching judgements for individual checks or systematic bias assessments.

Google Books, Scopus, Microsoft Academic and Mendeley for impact assessment of doctoral dissertations: A multidisciplinary analysis of the UK

A research doctorate normally culminates in publishing a dissertation reporting a substantial body of novel work. In the absence of a suitable citation index, this article explores the relative merits of alternative methods for the large-scale assessment of dissertation impact, using 150,740 UK doctoral dissertations from 2009–2018. Systematic methods for this were designed for Google Books, Scopus, Microsoft Academic, and Mendeley. Less than 1 in 8 UK doctoral dissertations had at least one Scopus (12%), Microsoft Academic (11%), or Google Books citation (9%), or at least one Mendeley reader (5%). These percentages varied substantially by subject area and publication year. Google Books citations were more common in the Arts and Humanities (18%), whereas Scopus and Microsoft Academic citations were more numerous in Engineering (24%). In the Social Sciences, Google Books (13%) and Scopus (12%) citations were important and in Medical Sciences, Scopus and Microsoft Academic citations to dissertations were rare (6%). Few dissertations had Mendeley readers (from 3% in Science to 8% in the Social Sciences) and further analysis suggests that Google Scholar finds more citations, but does not report information about all dissertations within a repository and is not a practical tool for large-scale impact assessment

Gender differences in citation impact for 27 fields and six English-speaking countries 1996–2014

Initiatives addressing the lack of women in many academic fields, and the general lack of senior women, need to be informed about the causes of any gender differences that may affect career progression, including citation impact. Previous research about gender differences in journal article citation impact has found the direction of any difference to vary by country and field, but has usually avoided discussions of the magnitude and wider significance of any differences and has not been systematic in terms of fields and/or time. This study investigates differences in citation impact between male and female first-authored research for 27 broad fields and six large English-speaking countries (Australia, Canada, Ireland, New Zealand, the UK, and the USA) from 1996 to 2014. The results show an overall female first author citation advantage, although in most broad fields it is reversed in all countries for some years. International differences include Medicine having a female first author citation advantage for all years in Australia, but a male citation advantage for most years in Canada. There was no general trend for the gender difference to increase or decrease over time. The average effect size is small, however, and unlikely to have a substantial influence on overall gender differences in researcher careers.

How common are explicit research questions in journal articles?

Although explicitly labeled research questions seem to be central to some fields, others do not need them. This may confuse authors, editors, readers, and reviewers of multidisciplinary research. This article assesses the extent to which research questions are explicitly mentioned in 17 out of 22 areas of scholarship from 2000 to 2018 by searching over a million full-text open access journal articles. Research questions were almost never explicitly mentioned (under 2%) by articles in engineering and physical, life, and medical sciences, and were the exception (always under 20%) for the broad fields in which they were least rare: computing, philosophy, theology, and social sciences. Nevertheless, research questions were increasingly mentioned explicitly in all fields investigated, despite a rate of 1.8% overall (1.1% after correcting for irrelevant matches). Other terminology for an article’s purpose may be more widely used instead, including aims, objectives, goals, hypotheses, and purposes, although no terminology occurs in a majority of articles in any broad field tested. Authors, editors, readers, and reviewers should therefore be aware that the use of explicitly labeled research questions or other explicit research purpose terminology is non-standard in most or all broad fields, although it is becoming less rare.

Large publishing consortia produce higher citation impact research but coauthor contributions are hard to evaluate

This paper introduces a simple agglomerative clustering method to identify large publishing consortia with at least 20 authors and 80% shared authorship between articles. Based on Scopus journal articles from 1996–2018, under these criteria, nearly all (88%) of the large consortia published research with citation impact above the world average, with the exceptions being mainly the newer consortia, for which average citation counts are unreliable. On average, consortium research had almost double (1.95) the world average citation impact on the log scale used (Mean Normalised Log Citation Score). At least partial alphabetical author ordering was the norm in most consortia. The 250 largest consortia were for nuclear physics and astronomy, involving expensive equipment, and for predominantly health-related issues in genomics, medicine, public health, microbiology and neuropsychology. For the health-related issues, except for the first and last few authors, authorship seem to primarily indicate contributions to the shared project infrastructure necessary to gather the raw data. It is impossible for research evaluators to identify the contributions of individual authors in the huge alphabetical consortia of physics and astronomy and problematic for the middle and end authors of health-related consortia. For small-scale evaluations, authorship contribution statements could be used when available.

Mendeley reader counts for US computer science conference papers and journal articles

Although bibliometrics are normally applied to journal articles when used to support research evaluations, conference papers are at least as important in fast-moving computing-related fields. It is therefore important to assess the relative advantages of citations and altmetrics for computing conference papers to make an informed decision about which, if any, to use. This paper compares Scopus citations with Mendeley reader counts for conference papers and journal articles that were published between 1996 and 2018 in 11 computing fields and that had at least one US author. The data showed high correlations between Scopus citation counts and Mendeley reader counts in all fields and most years, but with few Mendeley readers for older conference papers and few Scopus citations for new conference papers and journal articles. The results therefore suggest that Mendeley reader counts have a substantial advantage over citation counts for recently published conference papers due to their greater speed, but are unsuitable for older conference papers.

Author gender differences in psychology citation impact 1996-2018

Academic psychology in the USA is a gender success story in terms of overturning its early male dominance but there are still relatively few senior female psychology researchers. To assess whether there are gender differences in citation impact that might help to explain either of these trends, this study investigates psychology articles since 1996. Seven out of eight Scopus psychology categories had a majority of female first-authored journal articles by 2018. From regression analyses of first and last author gender and team size, female first authors associate with a slightly higher average citation impact, but extra authors have a 10 times stronger association with higher average citation impact. Last author gender has little association with citation impact. Female first authors are more likely to be in larger teams and if team size is attributed to the first author's work, then their apparent influence of female first authors on citation impact doubles. While gender differences in average citation impact are too small to account for gender-related trends in academic psychology, they warn that male-dominated citation-based ranking lists of psychologists do not reflect the state of psychology research today.