Comprehensive Researcher Achievement Model (CRAM): a framework for measuring researcher achievement, impact and influence derived from a systematic literature review of metrics and models

Research status and trends of the diabetic cardiomyopathy in the past 10 years (2012–2021): A bibliometric analysis

Background

Diabetic cardiomyopathy is one of the most life-threatening diabetic complications. However, the previous studies only discuss a particular aspect or characteristic of DCM, the current state and trends were explored by limited research. We aimed to perform a systemically bibliometric study of DCM research progress status in the past decade, visualize the internal conceptual structure and potential associations, and further explore the prospective study trends.

Methods

Articles related to DCM published from January 2012 to December 2021 were collected in the Web of Science core collection (WoSCC) database on June 24, 2022. We exported all bibliographic records, including titles, abstracts, keywords, authorship, institutions, addresses, publishing sources, references, citation times, and year of publication. In addition, the journal Impact Factor and Hirsch index were obtained from the Journal Citation Report. We conducted the data screening, statistical analysis, and visualization via the Bibliometrix R package. VOS viewer software was employed to generate the collaboration network map among countries and institutions for better performance in visualization.

Results

In total, 1,887 original research articles from 2012 to 2021 were identified. The number of annual publications rapidly increased from 107 to 278, and a drastic increase in citation times was observed in 2017–2019. As for global contributions, the United States was the most influential country with the highest international collaboration, while China was the most productive country. Professor Cai Lu was the most prolific author. Shandong University published the most articles. Cardiovascular Diabetology journal released the most DCM-related articles. “Metabolic Stress-induced Activation of FoxO1 Triggers Diabetic Cardiomyopathy in Mice” Battiprolu PK et al., J Clin Invest, 2012. was the most top-cited article regarding local citations. The top three keywords in terms of frequency were apoptosis, oxidative stress, and fibrosis. The analysis of future topic trends indicated that “Forkhead box protein O1,” “Heart failure with preserved ejection fraction,” “Dapagliflozin,” “Thioredoxin,” “Mitochondria dysfunction,” “Glucose,” “Pyroptosis,” “Cardiac fibroblast” and “Long non-coding RNA” could be promising hotspots.

Conclusion

This study provides meaningful insights into DCM, which is expected to assist cardiologists and endocrinologists in exploring frontiers and future research directions in the domain through a refined and concise summary.

#SoMe the Money! Value, Strategy, and Implementation of Social Media Engagement for Infectious Diseases Trainees, Clinicians, and Divisions

Social media platforms have revolutionized how we consume information, along with how to effectively present communication, education, and advocacy efforts. There is profound value in leveraging social media within these aspects for the field of infectious diseases, for divisions and individual clinicians. Herein, we provide the rationale to incorporate social media as a key competency for infectious diseases training and specific guidance on aspects of education and strategic development of new accounts critical for success.

Neurodegenerative Diseases and Cholesterol: Seeing the Field Through the Players

Neurodegenerative diseases, namely Alzheimer’s (AD), Parkinson’s (PD), and Huntington’s disease (HD) together with amyotrophic lateral sclerosis (ALS) and multiple sclerosis (MS), devastate millions of lives per year worldwide and impose an increasing socio-economic burden across nations. Consequently, these diseases occupy a considerable portion of biomedical research aiming to understand mechanisms of neurodegeneration and to develop efficient treatments. A potential culprit is cholesterol serving as an essential component of cellular membranes, as a cofactor of signaling pathways, and as a precursor for oxysterols and hormones. This article uncovers the workforce studying research on neurodegeneration and cholesterol using the TeamTree analysis. This new bibliometric approach reveals the history and dynamics of the teams and exposes key players based on citation-independent metrics. The team-centered view reveals the players on an important field of biomedical research.

TeamTree analysis: A new approach to evaluate scientific production

Advances in science and technology depend on the work of research teams and the publication of results through peer-reviewed articles representing a growing socio-economic resource. Current methods to mine the scientific literature regarding a field of interest focus on content, but the workforce credited by authorship remains largely unexplored. Notably, appropriate measures of scientific production are debated. Here, a new bibliometric approach named TeamTree analysis is introduced that visualizes the development and composition of the workforce driving a field. A new citation-independent measure that scales with the H index estimates impact based on publication record, genealogical ties and collaborative connections. This author-centered approach complements existing tools to mine the scientific literature and to evaluate research across disciplines.

Sex Differences in Academic Productivity Across Academic Ranks and Specialties in Academic Medicine: A Systematic Review and Meta-analysis

IMPORTANCE

Despite equal numbers of men and women entering medical school, women are underrepresented in the upper echelons of academic medicine and receive less compensation and research funding. Citation-related publication productivity metrics, such as the h-index, are increasingly used for hiring, salary, grants, retention, promotion, and tenure decisions. Exploring sex differences in these metrics across academic medicine provides deeper insight into why differences are observed in career outcomes.

OBJECTIVE

To systematically examine the available literature on sex differences in h-index of academic faculty physicians across all medical specialties and all levels of academic rank.

DATA SOURCES

Medical literature with the term h-index found in PubMed and published between January 1, 2009, and December 31, 2018, was used.

STUDY SELECTION

A PICOS (Population, Intervention, Comparison, and Outcomes), PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-analyses), and MOOSE (Meta-analysis of Observational Studies in Epidemiology) selection protocol was used to find observational studies that published h-indexes for faculty physicians that were stratified by sex. Studies were excluded if they were review articles, retracted, or unavailable online. Ultimately, 14 of 786 studies (1.78%) met the inclusion criteria.

DATA EXTRACTION AND SYNTHESIS

Data from 9 studies across 16 specialties were examined using weighted random-effects meta-analyses. Five studies were excluded because of overlapping specialties with another study or because they were missing appropriate statistics for the meta-analysis. Four of these studies were included in qualitative synthesis to bring the total to 13 studies.

MAIN OUTCOMES AND MEASURES

The primary study outcome was the h-index.

RESULTS

The meta-analysis included 10 665 North American unique academic physicians across 9 different studies from the years 2009 to 2018. Of the 10 665 physicians, 2655 (24.89%) were women. Summary effect sizes for mean h-indexes of men and women and mean h-index difference between men and women were determined for all faculty physicians and at each academic rank. Overall, female faculty had lower h-indexes than male faculty (mean difference, -4.09; 95% CI, -5.44 to -2.73; P < .001). When adjusting for academic rank, female faculty still had lower h-indexes than male faculty at the ranks of assistant professor (mean difference, -1.3; 95% CI, -1.90 to -0.72; P < .001), associate professor (mean difference, -2.09; 95% CI, -3.40 to -0.78; P = .002), and professor (mean difference, -3.41; 95% CI, -6.24 to -0.58; P = .02).

CONCLUSIONS AND RELEVANCE

In this systematic review and meta-analysis, women had lower h-indexes than men across most specialties and at all academic ranks, but it is unclear why these differences exist. These findings suggest that future investigation should be conducted regarding the causes of lower h-indexes in women and that interventions should be developed to provide a more equitable environment for all physicians regardless of sex.

Evaluating Scholars' Impact and Influence: Cross-sectional Study of the Correlation Between a Novel Social Media-Based Score and an Author-Level Citation Metric

BACKGROUND

The development of an author-level complementary metric could play a role in the process of academic promotion through objective evaluation of scholars' influence and impact.

OBJECTIVE

The objective of this study was to evaluate the correlation between the Healthcare Social Graph (HSG) score, a novel social media influence and impact metric, and the h-index, a traditional author-level metric.

METHODS

This was a cross-sectional study of health care stakeholders with a social media presence randomly sampled from the Symplur database in May 2020. We performed stratified random sampling to obtain a representative sample with all strata of HSG scores. We manually queried the h-index in two reference-based databases (Scopus and Google Scholar). Continuous features (HSG score and h-index) from the included profiles were summarized as the median and IQR. We calculated the Spearman correlation coefficients (ρ) to evaluate the correlation between the HSG scores and h-indexes obtained from Google Scholar and Scopus.

RESULTS

A total of 286 (31.2%) of the 917 stakeholders had a Google Scholar h-index available. The median HSG score for these profiles was 61.1 (IQR 48.2), and the median h-index was 14.5 (IQR 26.0). For the 286 subjects with the HSG score and Google Scholar h-index available, the Spearman correlation coefficient ρ was 0.1979 (P<.001), indicating a weak positive correlation between these two metrics. A total of 715 (78%) of 917 stakeholders had a Scopus h-index available. The median HSG score for these profiles was 57.6 (IQR 46.4), and the median h-index was 7 (IQR 16). For the 715 subjects with the HSG score and Scopus h-index available, ρ was 0.2173 (P<.001), also indicating a weak positive correlation.

CONCLUSIONS

We found a weak positive correlation between a novel author-level complementary metric and the h-index. More than a chiasm between traditional citation metrics and novel social media-based metrics, our findings point toward a bridge between the two domains.

Evaluating cancer research impact: lessons and examples from existing reviews on approaches to research impact assessment

BACKGROUND

Performing cancer research relies on substantial financial investment, and contributions in time and effort from patients. It is therefore important that this research has real life impacts which are properly evaluated. The optimal approach to cancer research impact evaluation is not clear. The aim of this study was to undertake a systematic review of review articles that describe approaches to impact assessment, and to identify examples of cancer research impact evaluation within these reviews.

METHODS

In total, 11 publication databases and the grey literature were searched to identify review articles addressing the topic of approaches to research impact assessment. Information was extracted on methods for data collection and analysis, impact categories and frameworks used for the purposes of evaluation. Empirical examples of impact assessments of cancer research were identified from these literature reviews. Approaches used in these examples were appraised, with a reflection on which methods would be suited to cancer research  impact evaluation going forward.

RESULTS

In total, 40 literature reviews were identified. Important methods to collect and analyse data for impact assessments were surveys, interviews and documentary analysis. Key categories of impact spanning the reviews were summarised, and a list of frameworks commonly used for impact assessment was generated. The Payback Framework was most often described. Fourteen examples of impact evaluation for cancer research were identified. They ranged from those assessing the impact of a national, charity-funded portfolio of cancer research to the clinical practice impact of a single trial. A set of recommendations for approaching cancer research impact assessment was generated.

CONCLUSIONS

Impact evaluation can demonstrate if and why conducting cancer research  is worthwhile. Using a mixed methods, multi-category assessment organised within a framework, will provide a robust evaluation, but the ability to perform this type of assessment may be constrained by time and resources. Whichever approach is used, easily measured, but inappropriate metrics should be avoided. Going forward, dissemination of the results of cancer research impact assessments will allow the cancer research community to learn how to conduct these evaluations.

Professional standards in bibliometric research evaluation? A meta-evaluation of European assessment practice 2005-2019

Despite growing demand for practicable methods of research evaluation, the use of bibliometric indicators remains controversial. This paper examines performance assessment practice in Europe-first, identifying the most commonly used bibliometric methods and, second, identifying the actors who have defined wide-spread practices. The framework of this investigation is Abbott's theory of professions, and I argue that indicator-based research assessment constitutes a potential jurisdiction for both individual experts and expert organizations. This investigation was conducted using a search methodology that yielded 138 evaluation studies from 21 EU countries, covering the period 2005 to 2019. Structured content analysis revealed the following findings: (1) Bibliometric research assessment is most frequently performed in the Nordic countries, the Netherlands, Italy, and the United Kingdom. (2) The Web of Science (WoS) is the dominant database used for public research assessment in Europe. (3) Expert organizations invest in the improvement of WoS citation data, and set technical standards with regards to data quality. (4) Citation impact is most frequently assessed with reference to international scientific fields. (5) The WoS classification of science fields retained its function as a de facto reference standard for research performance assessment. A detailed comparison of assessment practices between five dedicated organizations and other individual bibliometric experts suggests that corporate ownership and limited access to the most widely used citation databases have had a restraining effect on the development and diffusion of professional bibliometric methods during this period.

Flaws (and quality) in research today: can artificial intelligence intervene?

The existing flaws in both conducting and reporting of research have been outlined and criticized in the past. Weak research design, poor methodology, lack of fresh ideas and poor reporting are the main points to blame. Issues have been continually raised on the types of results published, review process, sponsorship, notion, ethics, and incentives in publishing, the role of regulatory agencies and stakeholders, the role of funding, and the cooperation between funders and academic institutions and the training of both clinicians and methodologists or statisticians. As a result, there is loss of the utmost goal: the production of robust research to form recommendations to support pragmatic decision in a real-world context. We propose the construction of a model based on artificial intelligence that could assist stakeholders, clinicians, and patients to guide conducting the best quality of research. We briefly describe the levels of the workflow, including the input and output data collection, the feature extraction/selection, the architecture, and parameterization of the model, along with its training, operation, and refinement.

Three Commonly Utilized Scholarly Databases and a Social Network Site Provide Different, But Related, Metrics of Pharmacy Faculty Publication

Scholarly productivity is a critical component of pharmacy faculty effort and is used for promotion and tenure decisions. Several databases are available to measure scholarly productivity; however, comparisons amongst these databases are lacking for pharmacy faculty. The objective of this work was to compare scholarly metrics from three commonly utilized databases and a social networking site focused on data from research-intensive colleges of pharmacy and to identify factors associated with database differences. Scholarly metrics were obtained from Scopus, Web of Science, Google Scholar, and ResearchGate for faculty from research-intensive (Carnegie Rated R1, R2, or special focus) United States pharmacy schools with at least two million USD in funding from the National Institutes of Health. Metrics were compared and correlations were performed. Regression analyses were utilized to identify factors associated with database differences. Significant differences in scholarly metric values were observed between databases despite the high correlations, suggestive of systematic variation in database reporting. Time since first publication was the most common factor that was associated with database differences. Google Scholar tended to have higher metrics than all other databases, while Web of Science had lower metrics relative to other databases. Differences in reported metrics between databases are apparent, which may be attributable to the time since first publication and database coverage of pharmacy-specific journals. These differences should be considered by faculty, reviewers, and administrative staff when evaluating scholarly performance.