How stars matter: Recruiting and peer effects in evolutionary biology

Data, measurement and empirical methods in the science of science

The advent of large-scale datasets that trace the workings of science has encouraged researchers from many different disciplinary backgrounds to turn scientific methods into science itself, cultivating a rapidly expanding 'science of science'. This Review considers this growing, multidisciplinary literature through the lens of data, measurement and empirical methods. We discuss the purposes, strengths and limitations of major empirical approaches, seeking to increase understanding of the field's diverse methodologies and expand researchers' toolkits. Overall, new empirical developments provide enormous capacity to test traditional beliefs and conceptual frameworks about science, discover factors associated with scientific productivity, predict scientific outcomes and design policies that facilitate scientific progress.

Comparison of scientometric achievements at PhD and scientific output ten years later for 4,790 academic researchers

Introduction

PhD is the highest awarded degree offered by universities in different disciplines. Owners of a PhD can teach at universities, start independent research and receive a higher salary while further building a scientific career. We examined whether the publication output before the PhD degree has a correlation with subsequent research activities.

Methods

We downloaded publication and citation data from the Hungarian Scientific Bibliography for Hungarian researchers who obtained PhD between the ages of 24 and 45. The researchers were grouped into eleven scientific sections. We examined the number of Q1 publications published in the previous 5 years, the H-index, the total number of citations for the last complete year, and the biological age of the researcher. Each parameter was computed for the year at which the PhD was obtained and ten years later. Pre-PhD publications (and citations for these) were excluded when assessing post-PhD track records. Spearman rank correlation and Kruskal-Wallis test were computed.

Results

We analyzed all together 4,790 researchers. We obtained a positive correlation between the number of Q1 publications before and after PhD (corr. coeff. = 0.21–0.54, p<0.01 in all sections), between the H-index before and after PhD (corr. coeff. = 0.32–0.56, p<0.01 in all sections), and between the citations received before and after PhD (corr. coeff. = 0.34–0.51, p<0.01 in all sections). All three metrics measured ten years after the PhD were negatively correlated with the age of the researcher at the time of obtaining the PhD (number of publications corr. coeff. = -0.09–0.22, p<0.05; H-index corr. coeff. = -0.09–0.29, p<0.08; number of citations corr. coeff. = -0.14–0.30, p<0.01). Among all disciplines, Philosophy and History and Engineering sciences show the strongest correlation between pre- and post-PhD output. When running multiple regression analysis for all three metrics as dependent variables and the number of articles, the H-index, the number of citations in the year of the PhD, the calendar year of PhD, and the gender of the researcher as independent variables, the number of articles and the H-index in the year of PhD reached the strongest positive correlations while gender had a negative correlation.

Conclusions

We independently evaluated pre- and post-PhD publication performance. In connection with age, the discipline-specific reference values of scientometric parameters at the time of obtaining the PhD can help to select candidates for postdoctoral grants and positions.

Reach for the stars: disentangling quantity and quality of inventors’ productivity in a multifaceted latent variable model

Star inventors generate superior innovation outcomes. Their capacity to invent high-quality patents might be decisive beyond mere productivity. However, the relationship between quantitative and qualitative dimensions has not been exhaustively investigated. The equal odds baseline (EOB) framework can explicitly model this relationship. This work combines a theoretical model for creative production with recent calls in the patentometrics literature for multifaceted measurement of the ability to create high-quality patents. The EOB is extended and analyzed through structural equation modeling. Specifically, we compared a multifaceted EOB model with a single latent variable for quality, and a two-dimensional model that distinguishes between technological complexity and value of invention portfolios. The two-dimensional model had better fit but weaker factor scores (for the “value” latent variable) than the unidimensional model. These findings suggest that both the uni- and the two-dimensional approaches can be directly used for extending research on star inventors, while for practical high-stakes assessments the two-dimensional model would require further improvements.

Comments are welcome

Scholars present their new research at seminars and conferences and send drafts to peers in hopes of receiving comments and suggestions that will improve the quality of their work. Using a dataset of projects that were initiated when authors were doing their doctoral studies, this article measures how much peers’ individual and collective comments improve the quality of research. Controlling for the quality of the research idea and author, I find that a one-standard-deviation increase in the number of peers’ individual and collective comments is associated with a 43% increase in the quality of the journal in which the project is published.

Do successful PhD outcomes reflect the research environment rather than academic ability?

Maximising research productivity is a major focus for universities world-wide. Graduate research programs are an important driver of research outputs. Choosing students with the greatest likelihood of success is considered a key part of improving research outcomes. There has been little empirical investigation of what factors drive the outcomes from a student's PhD and whether ranking procedures are effective in student selection. Here we show that, the research environment had a decisive influence: students who conducted research in one of the University's priority research areas and who had experienced, research-intensive, supervisors had significantly better outcomes from their PhD in terms of number of manuscripts published, citations, average impact factor of journals published in, and reduced attrition rates. In contrast, students’ previous academic outcomes and research training was unrelated to outcomes. Furthermore, students who received a scholarship to support their studies generated significantly more publications in higher impact journals, their work was cited more often and they were less likely to withdraw from their PhD. The findings suggest that experienced supervisors researching in a priority research area facilitate PhD student productivity. The findings question the utility of assigning PhD scholarships solely on the basis of student academic merit, once minimum entry requirements are met. Given that citations, publication numbers and publications in higher ranked journals drive university rankings, and that publications from PhD student contribute approximately one-third of all research outputs from universities, strengthening research infrastructure and supervision teams may be more important considerations for maximising the contribution of PhD students to a university’s international standing.

Corporate governance and the impact to the R&D lab

Purpose

The purpose of this paper is to determine whether the strength of corporate governance influences the firm’s ability to retain their key knowledge workers or inventors.

Design/methodology/approach

This paper links agency and innovation theory to develop the hypotheses. Agency theory predicts that the interests of employees are counter to those of firm owners. The authors predict that as shareholder power grows as corporate governance strengthens, inventors who are highly productive, and those who pursue risky but valuable exploratory innovation will leave the firm. Given prior scholarship in innovation theory establishing the critical contributions that new knowledge creation and exploratory innovation make to firms’ competitive advantage, the authors consider whether stronger firm-level corporate governance leads to the erosion of the firm’s competitive advantage. The hypotheses are empirically tested using generalized least squares estimation on a data set that combines data on firms, their patents and the governance provisions these firms adopt.

Findings

Using a 10-year sample of publicly traded US firms, the authors find that stronger corporate governance erodes the very foundation of a firm’s innovation capabilities. Stronger corporate governance reduces management job security, which makes managers more risk-averse. This heightened “managerial myopia” results in increased departures of highly valuable inventors employed by the firm. The authors show that these departing inventors are more productive inventors than those who remain and engage in more exploratory R&D than the remaining inventors at the firm.

Originality/value

The findings raise questions on the appropriateness of the adoption of governance provisions strengthening shareholder rights in firms pursuing innovation.

High research productivity in vertically undifferentiated higher education systems: Who are the top performers?

The growing scholarly interest in research top performers comes from the growing policy interest in research top performance itself. A question emerges: what makes someone a top performer? In this paper, the upper 10% of Polish academics in terms of research productivity are studied, and predictors of entering this class are sought. In the science system (and Poland follows global patterns), a small number of scholars produce most of the works and attract huge numbers of citations. Performance determines rewards, and small differences in talent translate into a disproportionate level of success, leading to inequalities in resources, research outcomes, and rewards. Top performers are studied here through a bivariate analysis of their working time distribution and their academic role orientation, as well as through a model approach. Odds ratio estimates with logistic regression of being highly productive Polish academics are presented. Consistently across major clusters of academic disciplines, the tiny minority of 10% of academics produces about half (44.7%) of all Polish publications (48.0% of publications in English and 57.2% of internationally co-authored publications). The mean research productivity of top performers across major clusters is on average 7.3 times higher than that of the other academics, and in terms of internationally co-authored publications, 12.07 times higher. High inequality was observed: the average research productivity distribution is highly skewed with a long tail on the right not only for all Polish academics but also for top performers. The class of top performers is as internally stratified as that of their lower-performing colleagues. Separate regression models for all academics, science, technology, engineering and mathematics academics, and social sciences and humanities academics are built based on a large national sample (2525 usable observations), and implications are discussed.