Measuring co-authorship and networking-adjusted scientific impact

Path of excellence: A co-authorship network analysis of European Research Council grant winners in social sciences

The international scientific community puts an ever-growing emphasis on research excellence and performance evaluation. So does the European Union with its flagship research excellence grant scheme organised by the European Research Council. This paper aims to provide an in-depth analysis of one of the ERC's thematic panels within the social sciences, namely the SH2 "Political Science" panel. The analysis is based on empirical, statistical methods, and network analysis tools to gain insights about the grant winners' publication patterns and their coauthor networks. The results draw up an academic career track of the grantees based on quantitative publication patterns and performance. Besides, a change in authorship can be observed, which is proven by the formation of new groups and intensifying intra-group collaboration patterns in the case of all three grant types. However, the ERC grant serves different functions for the winners of three different categories: for the Starting Grant winners, it offers the possibility to kick off and establish their research group, for the Consolidator Grant winners, it opens up new opportunities to extend their co-authorship network, and for the Advanced Grant winners, it offers the chance to start a new collaboration.

Exploring high scientific productivity in international co-authorship of a small developing country based on collaboration patterns

The number of published scientific paper grows rapidly each year, totaling more than 2.9 million annually. New methodologies and systems have been developed to analyze scientific production and performance indicators from large quantities of data available from the scientific databases, such as Web of Science or Scopus. In this paper, we analyzed the international scientific production and co-authorship patterns for the most productive authors from Serbia based on the obtained Web of Science dataset in the period 2006-2013. We performed bibliometric and scientometric analyses together with statistical and collaboration network analysis, to reveal the causes of extraordinary publishing performance of some authors. For such authors, we found significant inequality in distribution of papers over journals and countries of co-authors, using Gini coefficient and Lorenz curves. Most of the papers belong to multidisciplinary, interdisciplinary, and the field of applied sciences. We have discovered three specific collaboration patterns that lead to high productivity in international collaboration. First pattern corresponds to mega-authorship papers with hundreds of co-authors gathered in specific research groups. The other two collaboration patterns were found in mathematics and multidisciplinary science, mainly application of graph theory and computational methods in physical chemistry. The former pattern results in a star-shaped collaboration network with mostly individual collaborators. The latter pattern includes multiple actors with high betweenness centrality measure and identified brokerage roles. The results are compared with the later period 2014-2023, where high scientific production has been observed in some other fields, such as biology and food science and technology.

Impact of COVID-19 pandemic on researcher collaboration in business and economics areas on national level: a scientometric analysis

Purpose

The coronavirus disease 2019 (COVID-19) pandemic has greatly impacted society and academic life and research practices. This study is an attempt to comprehend whether a global emergency of COVID-19 pandemic has an impact on researcher international collaboration. The authors analyze the research collaboration before and after the beginning of the COVID-19 pandemic to understand how scientists collaborated within their own nation's borders and beyond.

Design/methodology/approach

The authors analyze the research collaboration before and after the beginning of the COVID-19 pandemic to understand how scientists collaborated within their own nation's borders and beyond. The authors collected a dataset of research publications published in journals in the research area of business and economics and indexed in the WoS Core Collection database by researchers from 11 countries (Austria, Denmark, Greece, Indonesia, Iran, Ireland, Korea (South), Mexico, Pakistan, Romania and Vietnam). In total, 14,824 publication records were considered for the literature analysis. This study presented the scientometric analysis of these publications using bibliometric, statistical, factor analysis and network analysis methods. The results are evaluated and interpreted in the context of the Hofstede's model of cultural dimensions. The results of this study provide evidence to research management to properly allocate their efforts to improve the researcher cooperation during the ongoing COVID-19 pandemic and to overcome its negative outcomes in the years to come.

Findings

The results of our study show that uncertainty avoidance as the cultural factor defined by the Hofstede's model has significantly influenced the properties of research collaboration networks in the domain of business and economics. Uncertainty avoidance focuses on how cultures adapt to changes and cope with uncertainty, while the global COVID-19 pandemic introduced a lot of change and uncertainty all levels of society around the world.

Research limitations/implications

The study exclusively examines 14,824 research outputs which have been indexed in the WoS Core Collection database from 2019 till 15 November 2020 and only covered one research area (business economics). Thus, documents published in any other different channels and sources which are not covered in WoS are excluded from this study. The authors have analyzed the publications from just 11 countries, which represent a small part of the global research output. Also, the Hofstede’s cultural dimensions model is not a unique way to study cultural characteristics at the national level.

Practical implications

The results of this study will provide evidence to research management to properly allocate their efforts to improve the researcher cooperation during the still ongoing COVID-19 pandemic and to overcome its negative outcomes in the years to come.

Originality/value

Considering the global impact and social distress due to the outbreak of COVID-19 pandemic, this study is significant in the present scenario for identifying the changes in the characteristics of research collaboration networks of 11 diverse (in terms of geographical distribution and cultural differences in terms of the Hofstede’s cultural dimensions model) countries between 2019 (the year before COVID-19) and 2020 (the year of COVID-19), which has not been done before.

Application of graph theory in the library domain—Building a faceted framework based on a literature review

Based on a literature review, we present a framework for structuring the application of graph theory in the library domain. Our goal is to provide both researchers and libraries with a standard tool to classify scientific work, at the same time allowing for the identification of previously underrepresented areas where future research might be productive. To achieve this, we compile graph theoretical approaches from the literature to consolidate the components of our framework on a solid basis. The extendable framework consists of multiple facets grouped into five categories whose elements can be arbitrarily combined. Libraries can benefit from these facets by using them as a point of reference for the (meta)data they offer. Further work on formally defining the framework’s categories as well as on integration of other graph-related research areas not discussed in this article (e.g. knowledge graphs) would be desirable and helpful in the future.

Seven decades of chemotherapy clinical trials: a pan-cancer social network analysis

Clinical trials establish the standard of cancer care, yet the evolution and characteristics of the social dynamics between the people conducting this work remain understudied. We performed a social network analysis of authors publishing chemotherapy-based prospective trials from 1946 to 2018 to understand how social influences, including the role of gender, have influenced the growth and development of this network, which has expanded exponentially from fewer than 50 authors in 1946 to 29,197 in 2018. While 99.4% of authors were directly or indirectly connected by 2018, our results indicate a tendency to predominantly connect with others in the same or similar fields, as well as an increasing disparity in author impact and number of connections. Scale-free effects were evident, with small numbers of individuals having disproportionate impact. Women were under-represented and likelier to have lower impact, shorter productive periods (P < 0.001 for both comparisons), less centrality, and a greater proportion of co-authors in their same subspecialty. The past 30 years were characterized by a trend towards increased authorship by women, with new author parity anticipated in 2032. The network of cancer clinical trialists is best characterized as strategic or mixed-motive, with cooperative and competitive elements influencing its appearance. Network effects such as low centrality, which may limit access to high-profile individuals, likely contribute to the observed disparities.

Predicting translational progress in biomedical research

Fundamental scientific advances can take decades to translate into improvements in human health. Shortening this interval would increase the rate at which scientific discoveries lead to successful treatment of human disease. One way to accomplish this would be to identify which advances in knowledge are most likely to translate into clinical research. Toward that end, we built a machine learning system that detects whether a paper is likely to be cited by a future clinical trial or guideline. Despite the noisiness of citation dynamics, as little as 2 years of postpublication data yield accurate predictions about a paper's eventual citation by a clinical article (accuracy = 84%, F1 score = 0.56; compared to 19% accuracy by chance). We found that distinct knowledge flow trajectories are linked to papers that either succeed or fail to influence clinical research. Translational progress in biomedicine can therefore be assessed and predicted in real time based on information conveyed by the scientific community's early reaction to a paper.

c-index and Subindices of the h-index: New Variants of the h-index to Account for Variations in Author Contribution

Objectives Bibliometrics are used to assess or compare the academic productivity of individuals or groups. Most of these metrics, including the widely used h-index, do not recognize the added contribution that is generally provided by authors listed first, second, second-to-last and last (enhanced positions) in a publication citation. We propose the c-index as a novel modification to the h-index that will better reflect an individual's academic output, incorporating authorship position. Methods One hundred and sixty-six academic neurosurgeons in eight New York City (NYC) metropolitan region training programs were identified through department websites. Using the Scopus citation database, bibliometric profiles were created for each surgeon. Once an individual's h-index was calculated, the h-core articles (those with h or more citations) were specifically assessed to determine citation author position. Novel bibliometric indices were created to reflect the number of h-core articles that accounted for primary (h_p), senior (h_s) or internal authorship (h_i) position. Weighted "involvement factors" for primary (i_p) and senior (i_s) author contribution were created to reflect the added value of "enhanced position" authorship in an individual's h-core publications. c-indices were created to reflect the author's h-index once augmented by primary (c_p), senior (c_s), and overall (c_o) "enhanced position" authorship. Comparisons were made within each institution and across institutions, according to academic rank (assistant professor, associate professor, professor and chairperson). Results Breakdown by academic rank showed an increasing average h-index progressing from assistant professor through professor rank with no significant difference demonstrated between professor and chair status. This pattern was seen across all departments (aggregate) but with fewer instances of significance at the level of individual departments. After h-index modification, c_p, c_s, and c_o indices showed a similarly significant trend. As faculty rank increased, there was a significant trend toward increasing numbers of articles with authors in enhanced positions and a higher percentage of articles with the author in a senior position. Academic faculty had higher h, c_p, and c_s indices than clinical faculty. Evaluation of each individual department revealed no significant trend regarding a department's higher average c_p or c_s. Average c-index for a department paralleled the average h-index of that department, with larger departments tending to have larger cumulative h, c_p, c_s, and c_o indices. No consistent correlation was seen between mean h-indices and academic rank at an individual departmental level. Conclusions This study examines the academic productivity of a subset of neurosurgical programs in the NYC metropolitan area as a test bed for novel bibliometric indices. h_p, h_i, and h_s represent the respective number of primary, internal and senior authorship papers that comprise an individual's h-core papers. c_p, c_s, and c_o, variations of the h-index metric, are designed to more accurately reflect the contributions by primary, secondary and senior authors. Increasing academic rank was associated with an increased number of articles with the author in enhanced positions and a higher percentage of articles in a senior position.

Dynamics of co-authorship and productivity across different fields of scientific research

We aimed to assess which factors correlate with collaborative behavior and whether such behavior associates with scientific impact (citations and becoming a principal investigator). We used the R index which is defined for each author as log(N_p)/log(I₁), where I₁ is the number of co-authors who appear in at least I₁ papers written by that author and N_p are his/her total papers. Higher R means lower collaborative behavior, i.e. not working much with others, or not collaborating repeatedly with the same co-authors. Across 249,054 researchers who had published ≥30 papers in 2000–2015 but had not published anything before 2000, R varied across scientific fields. Lower values of R (more collaboration) were seen in physics, medicine, infectious disease and brain sciences and higher values of R were seen for social science, computer science and engineering. Among the 9,314 most productive researchers already reaching N_p ≥ 30 and I₁ ≥ 4 by the end of 2006, R mostly remained stable for most fields from 2006 to 2015 with small increases seen in physics, chemistry, and medicine. Both US-based authorship and male gender were associated with higher values of R (lower collaboration), although the effect was small. Lower values of R (more collaboration) were associated with higher citation impact (h-index), and the effect was stronger in certain fields (physics, medicine, engineering, health sciences) than in others (brain sciences, computer science, infectious disease, chemistry). Finally, for a subset of 400 U.S. researchers in medicine, infectious disease and brain sciences, higher R (lower collaboration) was associated with a higher chance of being a principal investigator by 2016. Our analysis maps the patterns and evolution of collaborative behavior across scientific disciplines.

"Political co-authorships" in medical science journals

The issue of co-author relationships on medical sciences journal publications has become more pronounced as advances in technology have enabled collaboration across countries and institutions to occur much more efficiently. These relationships often have underlying political motivations and outcomes, including career advancement, attempting to increase prestige of a project, and maintaining research grants. Some authors may be listed as senior or honorary authors despite offering little or no contribution to the original research project. This may be done in an effort to enhance the gravitas of a research project, and attain publication in a highly regarded medical journal. The current review covers the topic of political co-authorship and germane literature and lists strategies to combat this phenomenon. Such co-authorship practices corrupt the integrity of the research process as they attempt to bypass the safeguard that medical journals and institutions have put in place to prevent fraud and falsification. A number of strategies have been proposed to combat the practice of co-authorship, but it may ultimately be an unavoidable feature of contemporary medical research publishing that is difficult to police. Clin. Anat. 30:831-834, 2017. © 2017Wiley Periodicals, Inc.

Author-paper affiliation network architecture influences the methodological quality of systematic reviews and meta-analyses of psoriasis

Moderate-to-severe psoriasis is associated with significant comorbidity, an impaired quality of life, and increased medical costs, including those associated with treatments. Systematic reviews (SRs) and meta-analyses (MAs) of randomized clinical trials are considered two of the best approaches to the summarization of high-quality evidence. However, methodological bias can reduce the validity of conclusions from these types of studies and subsequently impair the quality of decision making. As co-authorship is among the most well-documented forms of research collaboration, the present study aimed to explore whether authors’ collaboration methods might influence the methodological quality of SRs and MAs of psoriasis. Methodological quality was assessed by two raters who extracted information from full articles. After calculating total and per-item Assessment of Multiple Systematic Reviews (AMSTAR) scores, reviews were classified as low (0-4), medium (5-8), or high (9-11) quality. Article metadata and journal-related bibliometric indices were also obtained. A total of 741 authors from 520 different institutions and 32 countries published 220 reviews that were classified as high (17.2%), moderate (55%), or low (27.7%) methodological quality. The high methodological quality subnetwork was larger but had a lower connection density than the low and moderate methodological quality subnetworks; specifically, the former contained relatively fewer nodes (authors and reviews), reviews by authors, and collaborators per author. Furthermore, the high methodological quality subnetwork was highly compartmentalized, with several modules representing few poorly interconnected communities. In conclusion, structural differences in author-paper affiliation network may influence the methodological quality of SRs and MAs on psoriasis. As the author-paper affiliation network structure affects study quality in this research field, authors who maintain an appropriate balance between scientific quality and productivity are more likely to develop higher quality reviews.

Multiple Citation Indicators and Their Composite across Scientific Disciplines

Many fields face an increasing prevalence of multi-authorship, and this poses challenges in assessing citation metrics. Here, we explore multiple citation indicators that address total impact (number of citations, Hirsch H index [H]), co-authorship adjustment (Schreiber Hm index [Hm]), and author order (total citations to papers as single; single or first; or single, first, or last author). We demonstrate the correlation patterns between these indicators across 84,116 scientists (those among the top 30,000 for impact in a single year [2013] in at least one of these indicators) and separately across 12 scientific fields. Correlation patterns vary across these 12 fields. In physics, total citations are highly negatively correlated with indicators of co-authorship adjustment and of author order, while in other sciences the negative correlation is seen only for total citation impact and citations to papers as single author. We propose a composite score that sums standardized values of these six log-transformed indicators. Of the 1,000 top-ranked scientists with the composite score, only 322 are in the top 1,000 based on total citations. Many Nobel laureates and other extremely influential scientists rank among the top-1,000 with the composite indicator, but would rank much lower based on total citations. Conversely, many of the top 1,000 authors on total citations have had no single/first/last-authored cited paper. More Nobel laureates of 2011–2015 are among the top authors when authors are ranked by the composite score than by total citations, H index, or Hm index; 40/47 of these laureates are among the top 30,000 by at least one of the six indicators. We also explore the sensitivity of indicators to self-citation and alphabetic ordering of authors in papers across different scientific fields. Multiple indicators and their composite may give a more comprehensive picture of impact, although no citation indicator, single or composite, can be expected to select all the best scientists.

Citation indicators addressing total impact, co-authorship, and author positions offer complementary insights about impact. This article shows that a composite score including six citation indicators identifies extremely influential scientists better than single indicators.

Multiple citation indicators are used in science and scientific evaluation. With an increasing proportion of papers co-authored by many researchers, it is important to account for the relative contributions of different co-authors. We explored multiple citation indicators that address total impact, co-authorship adjustment, and author order (in particular, single, first, or last position authorships, since these positions suggest pivotal contributions to the work). We evaluated the top 30,000 scientists in 2013 based on each of six citation indicators (84,116 total scientists assessed) and also developed a composite score that combines the six indicators. Different scientists populated the top ranks when different indicators were used. Many Nobel laureates and other influential scientists rank among the top 1,000 with the composite indicator, but rank much lower based on total citations. Conversely, many of the top 1,000 authors on total citations had no single/first/last-authored cited paper. More Nobel laureates are among the top authors when authors are ranked by the composite score than by single indicators. Multiple indicators and their composite give a more comprehensive picture of impact, although no method can pick all the best scientists.

A generalized view of self-citation: direct, co-author, collaborative, and coercive induced self-citation

The phenomenon of self-citation can present in many different forms, including direct, co-author, collaborative, and coercive induced self-citation. It can also pertain to the citation of single scientists, groups of scientists, journals, and institutions. This article presents some case studies of extreme self-citation practices. It also discusses the implications of different types of self-citation. Self-citation is not necessarily inappropriate by default. In fact, usually it is fully appropriate but often it is even necessary. Conversely, inappropriate self-citation practices may be highly misleading and may distort the scientific literature. Coercive induced self-citation is the most difficult to discover. Coercive Induced self-citation may happen directly from reviewers of articles, but also indirectly from reviewers of grants, scientific advisors who steer a research agenda, and leaders of funding agencies who may espouse spending disproportionately large funds in research domains that perpetuate their own self-legacy. Inappropriate self-citation can be only a surrogate marker of what might be much greater distortions of the scientific corpus towards conformity to specific opinions and biases. Inappropriate self-citations eventually affect also impact metrics. Different impact metrics vary in the extent to which they can be gamed through self-citation practices. Citation indices that are more gaming-proof are available and should be more widely used. We need more empirical studies to dissect the impact of different types of inappropriate self-citation and to examine the effectiveness of interventions to limit them.

The Scientific Competitiveness of Nations

We use citation data of scientific articles produced by individual nations in different scientific domains to determine the structure and efficiency of national research systems. We characterize the scientific fitness of each nation—that is, the competitiveness of its research system—and the complexity of each scientific domain by means of a non-linear iterative algorithm able to assess quantitatively the advantage of scientific diversification. We find that technological leading nations, beyond having the largest production of scientific papers and the largest number of citations, do not specialize in a few scientific domains. Rather, they diversify as much as possible their research system. On the other side, less developed nations are competitive only in scientific domains where also many other nations are present. Diversification thus represents the key element that correlates with scientific and technological competitiveness. A remarkable implication of this structure of the scientific competition is that the scientific domains playing the role of “markers” of national scientific competitiveness are those not necessarily of high technological requirements, but rather addressing the most “sophisticated” needs of the society.

Fragmented Romanian sociology: growth and structure of the collaboration network

Structural patterns in collaboration networks are essential for understanding how new ideas, research practices, innovation or cooperation circulate and develop within academic communities and between and within university departments. In our research, we explore and investigate the structure of the collaboration network formed by the academics working full-time within all the 17 sociology departments across Romania. We show that the collaboration network is sparse and fragmented, and that it constitutes an environment that does not promote the circulation of new ideas and innovation within the field. Although recent years have witnessed an increase in the productivity of Romanian sociologists, there is still ample room for improvement in terms of the interaction infrastructure that ought to link individuals together so that they could maximize their potentials. We also fail to discern evidence in favor of the Matthew effect governing the growth of the network, which suggests scientific success and productivity are not rewarded. Instead, the structural properties of the collaboration network are partly those of a core-periphery network, where the spread of innovation and change can be explained by structural equivalence rather than by interpersonal influence models. We also provide support for the idea that, within the observed network, collaboration is the product of homophily rather than prestige effects. Further research on the subject based on data from other countries in the region is needed to place our results in a comparative framework, in particular to discern whether the behavior of the Romanian sociologist community is unique or rather common.

Ethics of using language editing services in an era of digital communication and heavily multi-authored papers

Scientists of many countries in which English is not the primary language routinely use a variety of manuscript preparation, correction or editing services, a practice that is openly endorsed by many journals and scientific institutions. These services vary tremendously in their scope; at one end there is simple proof-reading, and at the other extreme there is in-depth and extensive peer-reviewing, proposal preparation, statistical analyses, re-writing and co-writing. In this paper, the various types of service are reviewed, along with authorship guidelines, and the question is raised of whether the high-end services surpass most guidelines' criteria for authorship. Three other factors are considered. First, the ease of collaboration possible in the internet era allows multiple iterations between the author(s) and the "editing service", so essentially, papers can be co-written. Second, "editing services" often offer subject-specific experts who comment not only on the language, but interpret and improve scientific content. Third, the trend towards heavily multi-authored papers implies that the threshold necessary to earn authorship is declining. The inevitable conclusion is that at some point the contributions by "editing services" should be deemed sufficient to warrant authorship. Trying to enforce any guidelines would likely be futile, but nevertheless, it might be time to revisit the ethics of using some of the high-end "editing services". In an increasingly international job market, awareness of this problem might prove increasingly important in authorship disputes, the allocation of research grants, and hiring decisions.

Evaluating research and impact: a bibliometric analysis of research by the NIH/NIAID HIV/AIDS clinical trials networks

Evaluative bibliometrics uses advanced techniques to assess the impact of scholarly work in the context of other scientific work and usually compares the relative scientific contributions of research groups or institutions. Using publications from the National Institute of Allergy and Infectious Diseases (NIAID) HIV/AIDS extramural clinical trials networks, we assessed the presence, performance, and impact of papers published in 2006–2008. Through this approach, we sought to expand traditional bibliometric analyses beyond citation counts to include normative comparisons across journals and fields, visualization of co-authorship across the networks, and assess the inclusion of publications in reviews and syntheses. Specifically, we examined the research output of the networks in terms of the a) presence of papers in the scientific journal hierarchy ranked on the basis of journal influence measures, b) performance of publications on traditional bibliometric measures, and c) impact of publications in comparisons with similar publications worldwide, adjusted for journals and fields. We also examined collaboration and interdisciplinarity across the initiative, through network analysis and modeling of co-authorship patterns. Finally, we explored the uptake of network produced publications in research reviews and syntheses. Overall, the results suggest the networks are producing highly recognized work, engaging in extensive interdisciplinary collaborations, and having an impact across several areas of HIV-related science. The strengths and limitations of the approach for evaluation and monitoring research initiatives are discussed.

Is there a glass ceiling for highly cited scientists at the top of research universities?

University leaders aim to protect, shape, and promote the missions of their institutions. I evaluated whether top highly cited scientists are likely to occupy these positions. Of the current leaders of 96 U.S. high research activity universities, only 6 presidents or chancellors were found among the 4009 U.S. scientists listed in the ISIHighlyCited.com database. Of the current leaders of 77 UK universities, only 2 vice-chancellors were found among the 483 UK scientists listed in the same database. In a sample of 100 top-cited clinical medicine scientists and 100 top-cited biology and biochemistry scientists, only 1 and 1, respectively, had served at any time as president of a university. Among the leaders of 25 U.S. universities with the highest citation volumes, only 12 had doctoral degrees in life, natural, physical or computer sciences, and 5 of these 12 had a Hirsch citation index m < 1.0. The participation of highly cited scientists in the top leadership of universities is limited. This could have consequences for the research and overall mission of universities.

A bibliometric analysis of research in psychopharmacology by psychology departments (1987-2007)

From the very outset of scientific Psychology, psychologists have shown interest for drugs and their effects on behavior. This has given rise to numerous contributions, mostly in the form of Psychopharmacology publications. The aim of this study was to quantitatively evaluate these contributions and compare them with other academic disciplines related to Psychopharmacology. Using the PubMed database, we retrieved information about articles from 15 journals included in the Pharmacology and Pharmacy category of the Journal Citation Reports database for a 21-year period (1987 to 2007). There were 37540 articles which about 52% were represented by 3 journals. About 70% of psychology publications were represented by 2 of these journals. Psychology departments accounted for the 11% of the published papers, which places Psychology third behind Psychiatry and Pharmacology, which contributed to 22.69 and 13% respectively. Psychology contributed to the greatest number of studies in 3 journals, second in 3 and third in 8. This report represents the first effort to explore the contribution of academic Psychology to the multidisciplinary science of psychopharmacology. Although leaders of production of psychopharmacology research were from Psychiatry and Pharmacology, Psychology departments are an important source of studies and thus of knowledge in the field of Psychopharmacology.

The Hirsch h index in a non-mainstream area: methodology of the behavioral sciences in Spain

The h index has advantages over journal impact factors for assessing the research performance of individuals, and it is becoming a reference tool for career assessment that is starting to be considered by some agencies as an aid in decisions for promotion, allocation, and funding. The h index has been reported to have adequate properties as a measure of the research accomplishments of individuals in areas where h values are usually high (i.e., at or above 40), but some concerns have been raised that its validity in other non-mainstream research areas is suspect. This paper presents data from an exhaustive computation and analysis of h indices for 204 faculty members in the area of Methodology of the Behavioral Sciences in Spain, an area where h indices tend to be low worldwide. The results indicate that the h index is substantially increased by self-citations and that the average h of full professors is not meaningfully larger than the average h of associate professors. Other interesting relations between h indices and demographic and academic variables are described, including the gender and age bias of h. In this field, but perhaps also in other fields where the average h is low, little justification is found for the use of the h index as a fair measure of research performance that can aid in funding or promotion decisions.

Co-authorship network analysis: a powerful tool for strategic planning of research, development and capacity building programs on neglected diseases

Background

New approaches and tools were needed to support the strategic planning, implementation and management of a Program launched by the Brazilian Government to fund research, development and capacity building on neglected tropical diseases with strong focus on the North, Northeast and Center-West regions of the country where these diseases are prevalent.

Methodology/Principal Findings

Based on demographic, epidemiological and burden of disease data, seven diseases were selected by the Ministry of Health as targets of the initiative. Publications on these diseases by Brazilian researchers were retrieved from international databases, analyzed and processed with text-mining tools in order to standardize author- and institution's names and addresses. Co-authorship networks based on these publications were assembled, visualized and analyzed with social network analysis software packages. Network visualization and analysis generated new information, allowing better design and strategic planning of the Program, enabling decision makers to characterize network components by area of work, identify institutions as well as authors playing major roles as central hubs or located at critical network cut-points and readily detect authors or institutions participating in large international scientific collaborating networks.

Conclusions/Significance

Traditional criteria used to monitor and evaluate research proposals or R&D Programs, such as researchers' productivity and impact factor of scientific publications, are of limited value when addressing research areas of low productivity or involving institutions from endemic regions where human resources are limited. Network analysis was found to generate new and valuable information relevant to the strategic planning, implementation and monitoring of the Program. It afforded a more proactive role of the funding agencies in relation to public health and equity goals, to scientific capacity building objectives and a more consistent engagement of institutions and authors from endemic regions based on innovative criteria and parameters anchored on objective scientific data.

The selection and prioritization of research proposals is always a challenge, particularly when addressing neglected tropical diseases, as the scientific communities are relatively small, funding is usually limited and the disparity between the science and technology capacity of different countries and regions is enormous. When the Ministry of Health and the Ministry of Science and Technology of Brazil decided to launch an R&D program on neglected diseases for which at least 30% of the Program's resources were supposed to be invested in institutions and authors from the poorest regions of Brazil, it became clear to us that new strategies and approaches would be required. Social network analysis of co-authorship networks is one of the new approaches we are exploring to develop new tools to help policy-/decision-makers and academia jointly plan, implement, monitor and evaluate investments in this area. Publications retrieved from international databases provide the starting material. After standardization of names and addresses of authors and institutions with text mining tools, networks are assembled and visualized using social network analysis software. This study enabled the development of innovative criteria and parameters, allowing better strategic planning, smooth implementation and strong support and endorsement of the Program by key stakeholders.

Looking for landmarks: the role of expert review and bibliometric analysis in evaluating scientific publication outputs

OBJECTIVE

To compare expert assessment with bibliometric indicators as tools to assess the quality and importance of scientific research papers.

METHODS AND MATERIALS

Shortly after their publication in 2005, the quality and importance of a cohort of nearly 700 Wellcome Trust (WT) associated research papers were assessed by expert reviewers; each paper was reviewed by two WT expert reviewers. After 3 years, we compared this initial assessment with other measures of paper impact.

RESULTS

Shortly after publication, 62 (9%) of the 687 research papers were determined to describe at least a 'major addition to knowledge' -6 were thought to be 'landmark' papers. At an aggregate level, after 3 years, there was a strong positive association between expert assessment and impact as measured by number of citations and F1000 rating. However, there were some important exceptions indicating that bibliometric measures may not be sufficient in isolation as measures of research quality and importance, and especially not for assessing single papers or small groups of research publications.

CONCLUSION

When attempting to assess the quality and importance of research papers, we found that sole reliance on bibliometric indicators would have led us to miss papers containing important results as judged by expert review. In particular, some papers that were highly rated by experts were not highly cited during the first three years after publication. Tools that link expert peer reviews of research paper quality and importance to more quantitative indicators, such as citation analysis would be valuable additions to the field of research assessment and evaluation.