"Publish or Perish" Promotes Medical Literature Quantity Over Quality

Medical researchers' perceptions regarding research evaluation: a web-based survey in Japan

OBJECTIVES

Japanese medical academia continues to depend on quantitative indicators, contrary to the general trend in research evaluation. To understand this situation better and facilitate discussion, this study aimed to examine how Japanese medical researchers perceive quantitative indicators and qualitative factors of research evaluation and their differences by the researchers' characteristics.

DESIGN

We employed a web-based cross-sectional survey and distributed the self-administered questionnaire to academic society members via the Japanese Association of Medical Sciences.

PARTICIPANTS

We received 3139 valid responses representing Japanese medical researchers in any medical research field (basic, clinical and social medicine).

OUTCOMES

The subjective importance of quantitative indicators and qualitative factors in evaluating researchers (eg, the journal impact factor (IF) or the originality of the research topic) was assessed on a four-point scale, with 1 indicating 'especially important' and 4 indicating 'not important'. The attitude towards various opinions in quantitative and qualitative research evaluation (eg, the possibility of research misconduct or susceptibility to unconscious bias) was also evaluated on a four-point scale, ranging from 1, 'strongly agree', to 4, 'completely disagree'.

RESULTS

Notably, 67.4% of the medical researchers, particularly men, younger and basic medicine researchers, responded that the journal IF was important in researcher evaluation. Most researchers (88.8%) agreed that some important studies do not get properly evaluated in research evaluation using quantitative indicators. The respondents perceived quantitative indicators as possibly leading to misconduct, especially in basic medicine (strongly agree-basic, 22.7%; clinical, 11.7%; and social, 16.1%). According to the research fields, researchers consider different qualitative factors, such as the originality of the research topic (especially important-basic, 46.2%; social, 39.1%; and clinical, 32.0%) and the contribution to solving clinical and social problems (especially important-basic, 30.4%; clinical, 41.0%; and social, 52.0%), as important. Older researchers tended to believe that qualitative research evaluation was unaffected by unconscious bias.

CONCLUSION

Despite recommendations from the Declaration on Research Assessment and the Leiden Manifesto to de-emphasise quantitative indicators, this study found that Japanese medical researchers have actually tended to prioritise the journal IF and other quantitative indicators based on English-language publications in their research evaluation. Therefore, constantly reviewing the research evaluation methods while respecting the viewpoints of researchers from different research fields, generations and genders is crucial.

Editorial Commentary: Published Literature on Platelet-Rich Plasma Is Extensive but Flawed in Methodology

Recent research shows that most orthopaedic published investigations on platelet-rich plasma (PRP) lack reporting of preparation methods or PRP composition. The studies are thus unreproducible and the outcomes cannot be verified. Unfortunately, this is not surprising. A PubMed search reveals more than 500 orthopaedic PRP publications from 2023 alone, almost 42 musculoskeletal PRP papers per month, and unfortunately, despite peer review, Nature reports that 44% of published research is flawed and 26% of studies cannot be trusted. PubMed citation does not ensure quality. Guidelines and checklists such as the CONSORT statement (Consolidated Standards of Reporting Trails) can result in quality.

The confidence in the results of physiotherapy systematic reviews in the musculoskeletal field is not increasing over time: a meta-epidemiological study using AMSTAR 2 tool

OBJECTIVES

To assess the confidence in the results of systematic reviews on the effectiveness of physiotherapy for musculoskeletal conditions in the past 10 years and to analyze trends and factors associated.

METHODS

This is a metaepidemiological study on systematic reviews (SRs) with meta-analysis of randomized controlled trials (RCTs). MEDLINE, Cochrane Database of Systematic Reviews, CINAHL, and PEDro were searched for SRs of RCT on physiotherapy interventions for musculoskeletal disorders from December 2012 to December 2022. Two researchers independently screened the records based on the inclusion criteria; a random sample of 100 studies was selected, and each journal, author, and study variable was extracted. The methodological quality of SRs was independently assessed with the AMSTAR 2 tool. Any disagreement was solved by consensus.

RESULTS

The confidence in SRs results was critically low in 90% of the studies, and it did not increase over time. Cochrane reviews are predominantly represented in the higher AMSTAR 2 confidence levels, with a statistically significant difference compared to non-Cochrane reviews. The last author's H-index is the only predictor of higher confidence among the variables analyzed (OR 1.04; 95% CI: 1.01, 1.06).

CONCLUSION

The confidence in SRs results is unacceptably low. Given the relevance of musculoskeletal disorders and the impact of evidence synthesis on the clinical decision-making process, there is an urgent need to improve the quality of secondary research by adopting more rigorous methods.

Detecting generative artificial intelligence in scientific articles: Evasion techniques and implications for scientific integrity

BACKGROUND

Artificial intelligence (AI) tools, although beneficial for data collection and analysis, can also facilitate scientific fraud. AI detectors can help resolve this problem, but their effectiveness depends on their ability to track AI progress. In addition, many methods of evading AI detection exist and their constantly evolving sophistication can make the task more difficult. Thus, from an AI-generated text, we wanted to: (1) evaluate the AI detection sites on a text generated entirely by the AI, (2) test the methods described for evading AI detection, and (3) evaluate the effectiveness of these methods to evade AI detection on the sites tested previously.

HYPOTHESIS

Not all AI detection tools are equally effective in detecting AI-generated text and some techniques used to evade АI detection can make an AI-produced text almost undetectable.

MATERIALS AND METHODS

We created a text with ChatGPT-4 (Chat Gеnеrаtivе Prе-trained Transformer) and submitted it to 11 АI detection web tools (Оriginаlity, ZеrоGPT, Writеr, Cоpylеаks, Crоssplag, GPTZеrо, Sapling, Cоntеnt аt scаlе, Cоrrеctоr, Writеfull еt Quill), bеfоrе аnd аftеr applying strategies to minimise AI detection. The strategies used to minimize AI detection were the improvement of command messages in ChatPGT, the introduction of minor grammatical errors such as comma deletion, paraphrasing, and the substitution of Latin letters with similar Cyrillic letters (a and о) which is also a method used elsewhere to evade the detection of plagiarism. We have also tested the effectiveness of these tools in correctly identifying a scientific text written by a human in 1960.

RESULTS

From the initial text generated by the AI, 7 of the 11 detectors concluded that the text was mainly written by humans. Subsequently, the introduction of simple modifications, such as the removal of commas or paraphrasing can effectively reduce AI detection and make the text appear human for all detectors. In addition, replacing certain Latin letters with Cyrillic letters can make an AI text completely undetectable. Finally, we observe that in a paradoxical way, certain sites detect a significant proportion of AI in a text written by a human in 1960.

DISCUSSION

AI detectors have low efficiency, and simple modifications can allow even the most robust detectors to be easily bypassed. The rapid development of generative AI raises questions about the future of scientific writing but also about the detection of scientific fraud, such as data fabrication.

LEVEL OF EVIDENCE

III Control case study.

Validation of publication pressure questionnaire-revised into Chinese nurses from a university hospital

AIMS

To translate and validate the Publication Pressure Questionnaire-revised (PPQ-r) among the Chinese nurses.

DESIGN

A cross-sectional descriptive study.

METHODS

The classical forward/backward translation of the PPQ-r was conducted first, and then the psychometric values were tested using the data collected in September, 2021 among 924 Chinese nurses from a university hospital in China by calculating its content validity index, construct validity, both convergent and discriminant validity, and internal consistency reliability following the COSMIN guidelines.

RESULTS

The proposed three-factor structure of original PPQ-r cannot be confirmed in present study. With exploratory factor analysis, a new two-factor structure with 9 items was extracted, accounting for 62.718% of the variance. Confirmatory factor analysis indicated satisfactory goodness of all fit index values: χ2  = 75.53; df = 23; χ2 /df = 3.28; CFI = 0.973; TLI = 0.958; RMSEA = 0.07 and SRMR = 0.047. Both sub-scales yielded good internal consistency with Cronbach's alpha being 0.845 and 0.876, respectively. The Chinese version of PPQ-r could be accepted as a reliable and valid tool to measure the Chinese nurses' publication pressure.

PATIENT OR PUBLIC CONTRIBUTION

Participation of nurses in this study was limited to the data provided through participant survey.

Strategies to Mitigate Against Least-Publishable Units in Medical and Scientific Research Publications

Least-publishable units, aka minimal publishable units, smallest publishable units, fractions of scholarly effort, and "salami slicing" divide a single research publication into a number of papers with small amounts of information in each paper. This results in quantity rather than quality; is ethically inappropriate; creates extra work for readers, future authors, reviewers, and editors; and can result in redundancy, self-plagiarism, publication overlap, and duplicate reporting of patient data that can result in inaccurate conclusions in systematic reviews. Increased awareness and actionable intervention can help to reverse this growing trend.

Subgroup analysis of scientific performance in the field of arthroplasty

Introduction

Arthroplasty is the final treatment option for maintaining mobility and quality of life in many primary degenerative and (post-) traumatic joint diseases. Identification of research output and potential deficits for specific subspecialties may be an important measure to achieve long-term improvement of patient care in this field.

Methods

Using specific search terms and Boolean operators, all studies published since 1945 to the subgroups of arthroplasty listed in the Web of Science Core Collection were included. All identified publications were analysed according to bibliometric standards, and comparative conclusions were drawn regarding the scientific merit of each subgroup.

Results

Most publications investigated the subgroups of septic surgery and materials followed by approach, navigation, aseptic loosening, robotic and enhanced recovery after surgery (ERAS). In the last 5 years, research in the fields of robotic and ERAS achieved the highest relative increase in publications In contrast, research on aseptic loosening has continued to lose interest over the last 5 years. Publications on robotics and materials received the most funding on average while those on aseptic loosening received the least. Most publications originated from USA, Germany, and England, except for research on ERAS in which Denmark stood out. Relatively, publications on aseptic loosening received the most citations, whereas the absolute scientific interest was highest for the topic infection.

Discussion

In this bibliometric subgroup analysis, the primary scientific outputs focused on septic complications and materials research in the field of arthroplasty. With decreasing publication output and the least financial support, intensification of research on aseptic loosening is urgently recommended.

Generation Learning Differences in Surgery: Why They Exist, Implication, and Future Directions

The evolution of the knowledge economy and technology industry have fundamentally changed the learning environments occupied by contemporary surgical trainees and created pressures that will force the surgical community to consider. Although some learning differences are intrinsic to the generations themselves, these differences are primarily a function of the environments in which surgeons of different generations trained. Acknowledgment of the principles of connectivism and thoughtful integration of artificial intelligence and computerized decision support tools must play a central role in charting the future course of surgical education.

Retractions in Dermatology Literature Between 1982 and 2022: Cross-sectional Study

Collapse

Assessing Factors Associated With Altmetric Attention Score: A Preliminary Study of 3 Hand Surgery Journals

BACKGROUND

Traditional measures of evaluating scholarly output do not capture the impact social media can provide in disseminating and promoting research. We sought to better understand the level of online attention that high-quality hand research received.

METHODS

Scientific manuscripts published from 2017 in Journal of Hand Surgery (American Volume) (JHS-A), Journal of Hand Surgery (European Volume) (JHS-E), and HAND were recorded. Manuscript characteristics were determined, including the number of citations. Altmetric Attention Score (AAS), a measure of a manuscript's online attention and impact, was determined, as well as Twitter mentions, Facebook mentions, and news outlet mentions. Spearman rho (ρ) correlation coefficient was used to evaluate the relationship between the number of citations and AAS. Multivariable linear regression analysis was performed to determine which manuscript factors were associated with AAS.

RESULTS

A total of 323 manuscripts were identified. There was a weak positive correlation between the number of citations and AAS; however, this relationship did not exist for each individual journal. Publication in HAND and JHS-E were associated with lower average manuscript AAS when using JHS-A as the reference group. Two additional factors were also associated with increased manuscript AAS: (1) being a clinical study focused on a specific upper extremity anatomical location; and (2) increasing number of institutions on a study.

CONCLUSIONS

Publication in HAND and JHS-E were associated with lower manuscript AAS when using JHS-A as the reference group, suggesting that HAND and JHS-E have room for improvement in using social media to share their high-quality hand surgery scientific articles.

The Author Impact Factor as a Metric to Evaluate the Impact of Neurosurgical Researchers

OBJECTIVE

The objective of this study was to assess the Author Impact Factor (AIF) as a useful metric and as a complement to the h-index among neurosurgical researchers.

METHODS

The 5-year AIF and h-index were compared among 3 groups of researchers: 1) the 100 most prolific of all time within general neurosurgical journals ("Experienced"), 2) the 100 most prolific during the 2015-2019 period within general neurosurgical journals ("Trending Group"), and 3) the 100 postgraduation year 7 neurosurgical residents with the highest h-index ("Amateur").

RESULTS

The Amateur group had a lower median h-index than the Experienced (6 vs. 55; P < 0.001) and Trending (6 vs. 43; P < 0.001) groups. The highest h-index of the Amateur group (24) was lower than the first quartile of the Experienced (46.25) and Trending (26.00) groups. The Amateur group had a lower median 5-year AIF than the Experienced (2.15 vs. 3.17; P < 0.001) and Trending (2.15 vs. 2.85; P = 0.02) groups. Unlike the h-index, the gap between the 5-year AIF distribution of the Amateur group and other groups was not profound. Although there was a positive correlation between the metrics in the 3 groups, they did not proxy for each other. For instance, while the h-index of some experienced authors that have not published recently was high, their AIFs were zero. Also, some Amateur authors published very impactful articles and had a high 5-year AIF. However, since their number of publications is inevitably low, their h-index were low.

CONCLUSIONS

The AIF provides intuitive and complementary information to the h-index regarding the research output of neurosurgical authors.

Biosynthetic versatility of marine-derived fungi on the delivery of novel antibacterial agents against priority pathogens

Despite the increasing number of novel marine natural products being reported from fungi in the last three decades, to date only the broad-spectrum cephalosporin C can be tracked back as marine fungal-derived drug. Cephalosporins were isolated in the early 1940s from a strain of Acremonium chrysogenum obtained in a sample collected in sewage water in the Sardinian coast, preliminary findings allowing the discovery of cephalosporin C. Since then, bioprospection of marine fungi has been enabling the identification of several metabolites with antibacterial effects, many of which proving to be active against multi-drug resistant strains, available data suggesting also that some might fuel the pharmaceutical firepower towards some of the bacterial pathogens classified as a priority by the World Health Organization. Considering the success of their terrestrial counterparts on the discovery and development of several antibiotics that are nowadays used in the clinical setting, marine fungi obviously come into mind as producers of new prototypes to counteract antibiotic-resistant bacteria that are no longer responding to available treatments. We mainly aim to provide a snapshot on those metabolites that are likely to proceed to advanced preclinical development, not only based on their antibacterial potency, but also considering their targets and modes of action, and activity against priority pathogens.

Do Year-Out Programs Make Medical Students More Competitive Candidates for Orthopedic Surgery Residencies?

OBJECTIVE

As orthopedic surgery residency programs are becoming more competitive, medical students interested in orthopedics are increasingly completing "year-out" programs. This study sought to evaluate student and faculty perceptions of these programs.

DESIGN

A survey evaluating baseline characteristics was disseminated to directors of year-out programs identified through postings on the orthopedics forum, Orthogate.org. A second survey was sent to all program directors (PDs) of accredited US orthopedic residencies, while a final survey was distributed to participants identified by year-out PDs.

SETTING

Ninety-six orthopedic year-out programs at 56 institutions were contacted.

PARTICIPANTS

Twenty-six year-out programs, 72 PDs of ACGME-accredited orthopedic residencies, and 34 year-out participants from 6 programs completed our questionnaires.

RESULTS

73.1% (19) year-out program provided funding to participants, averaging $30,368. 84.6% (22) reported >75% match rates into orthopedics for participants. 65.4% (17) of programs selected students between their MS3/MS4 school years. 4.2% (3) of residency PDs agreed or strongly agreed that year-out programs were important factors for consideration in residency programs, compared with 82.4% (28) of year-out participants and 69.2% (18) of year-out PDs (p < 0.001). 58.8% (2) of year-out participants cited completion of a year-out for improving the chance of matching into any orthopedic residency, while 85.3% (29) wanted to be more competitive for top programs. The average Step 1 score was 248, which was insignificantly different from the national average for matched orthopedic applicants.

CONCLUSIONS

Orthopedic year-out programs have dramatically increased in number over the last 20 years. Most of these programs are funded, 1-year clinical research fellowships with relative match success for participants pursuing orthopedic residencies. While year-out PDs and students consider participation in such programs to be an important factor for residency applications, and often participate in them in order to improve their competitiveness for matching at desired programs, residency PDs overall hold different views.

Bridging the Gap with Clinicians: The Issue of Underrecognition of Pathologists and Radiologists as Scientific Authors in Contemporary Medical Literature

The purpose of this study was to evaluate recognition of pathologists and radiologists as coauthors in case reports in the field of surgical oncology. The MEDLINE database was searched for all full free text case reports involving human material published from April 1, 2011 until March 31, 2016, using search terms: "case report" + "tumors" + "surgery" + "malignant". The search strategy identified a total of 1427 case reports of which 907 were included in this analysis. Of 807 articles with histopathological images and/or descriptions, 352 (43.6%) did not acknowledge or include the pathologist as a coauthor. Of 662 case reports with radiographic images and/or their description, 537 (81.1%) did not list the radiologist as coauthor nor acknowledge them. In case reports containing histopathological images, significantly more pathologists were either listed as coauthors or acknowledged compared to those who were not (Z = 5.128; p = 0.001). However, among case reports containing radiographic images, there were significantly less articles either listing radiologists as coauthors or acknowledging them compared to a larger proportion of articles in which radiologists were omitted (Z = - 22.646; p = 0.001). In conclusion, pathologists and radiologists are underrecognized as coauthors in surgical oncology case reports in spite of obvious proof of their contribution to manuscript preparation. When involved in research and publishing, all physicians should be aware of fair and honest collaboration with specialists in other clinical and non-clinical disciplines to better serve the scientific community.

Why we publish where we do: Faculty publishing values and their relationship to review, promotion and tenure expectations

Using an online survey of academics at 55 randomly selected institutions across the US and Canada, we explore priorities for publishing decisions and their perceived importance within review, promotion, and tenure (RPT). We find that respondents most value journal readership, while they believe their peers most value prestige and related metrics such as impact factor when submitting their work for publication. Respondents indicated that total number of publications, number of publications per year, and journal name recognition were the most valued factors in RPT. Older and tenured respondents (most likely to serve on RPT committees) were less likely to value journal prestige and metrics for publishing, while untenured respondents were more likely to value these factors. These results suggest disconnects between what academics value versus what they think their peers value, and between the importance of journal prestige and metrics for tenured versus untenured faculty in publishing and RPT perceptions.

Do Some Patients Count More Than Others? Reporting Outcomes of the Same Patient in More Than One Study Requires Disclosure

Clinical databases allow researchers to test multiple hypotheses. This could result in including outcomes on the same patient or patients in more than 1 study. When this occurs, it is vital for authors to clarify multiple reporting in their study methods to avoid having patients counted more than once in future systematic reviews or meta-analyses. As a caveat, primary authors should consider whether publication of multiple studies is important or whether they are simply generating "least publishable units" (LPUs, also known as salami slicing).

Oxygen-Ozone Therapy for the Treatment of Knee Osteoarthritis: A Systematic Review of Randomized Controlled Trials

PURPOSE

To review the available literature on the application of oxygen-ozone therapy (OOT) in the treatment of knee osteoarthritis (KOA) to understand its therapeutic potential and to compare it with other conservative treatment options.

METHODS

A systematic review of the literature was performed on the PubMed, Cochrane, Embase, ResearchGate, and PedRo Databases, with the following inclusion criteria: (1) randomized controlled trials (RCTs), (2) written in English, (3) published on indexed journals in the last 20 years (1998-2018), (4) dealing with the use of ozone intra-articular injection for the treatment of KOA. The risk of bias was assessed by the Cochrane Risk of Bias tool for RCTs.

RESULTS

Eleven studies involving 858 patients in total (629 female and 229 male) were included. Patients in the control groups received different treatments: placebo in 1 trial; hyaluronic acid in 2 studies; hyaluronic acid and PRP in 1 trial; corticosteroids in 4; and hypertonic dextrose, radiofrequency, or celecoxib + glucosamine in the remaining 3 trials. In looking at the quality of the available literature, we found that none of the studies included reached "good quality" standard, 2 were ranked as "fair," and the rest were considered "poor." No major complications or serious adverse events were reported following intra-articular OOT, which provided encouraging pain relief at short term. On the basis of the available data, no clear indication emerged from the comparison of OOT with other established treatments for KOA.

CONCLUSIONS

The analysis of the available RCTs on OOT for KOA revealed poor methodologic quality, with most studies flawed by relevant bias, thus severely limiting the possibility of drawing conclusions on the efficacy of OOT compared with other treatments. On the basis of the data available, OOT has, however, proven to be a safe approach with encouraging effects in pain control and functional recovery in the short-middle term.

LEVEL OF EVIDENCE

Systematic review of Level I and III studies.

Arthroscopy, Sports Medicine, and Rehabilitation (ASMAR): A New, Open Access, Electronic Journal

Arthroscopy, Sports Medicine, and Rehabilitation (ASMAR) is our new, open-access, electronic, companion journal to Arthroscopy and Arthroscopy Techniques. While there is already "so much to read, so little time," bearing in mind the context that some submissions are not accepted for publication in Arthroscopy despite admirable quality, the necessity of ASMAR becomes clear.