Characteristics of Orthopedic Publications in High-Impact General Medical Journals

Comparing Orthopaedic Randomized Control Trials Published in High-Impact Medical and Orthopaedic Journals

INTRODUCTION

Orthopaedic studies published in high-impact medical journals are often believed to have a high prevalence of negative or neutral results and possess methodological characteristics that may bias toward nonsurgical treatments. The purpose of this study was to compare study characteristics, methodologic quality, exposure, and outcome direction among orthopaedic randomized control trials (RCTs) published in high-impact medical and orthopaedic journals and to identify study attributes associated with greater impact.

METHODS

RCTs published between January 2010 and December 2020 in the five medical journals and 10 orthopaedic journals with the highest 5-year impact factors were analyzed. Inclusion criteria were RCTs reporting on orthopaedic surgical intervention compared with nonsurgical or less-invasive surgical procedures. Study characteristics, methodologic quality (Jadad scale), outcomes, and altmetric data were collected. Primary outcomes were categorized as positive (favoring surgical/more-extensive surgery), negative (favoring nonsurgical/less-extensive surgery), or neutral.

RESULTS

One hundred twenty-eight RCTs were analyzed; 26 from medical and 102 from orthopaedic journals. Studies published in medical journals included more authors ( P < 0.001), larger sample sizes ( P < 0.001), more institutions ( P < 0.001), and more often received funding ( P < 0.001). The average Jadad scale did not significantly differ between journals ( P = 0.14). The direction of the primary study outcome did not differ between journals ( P = 0.22). Average AAS and annual citation rates were higher in RCTs published in medical journals ( P < 0.001). Publication in a medical journal was the only covariate associated with higher annual citation rates ( P < 0.001) and AAS ( P < 0.001) on multivariable analyses.

DISCUSSION

High-impact medical journals do not publish orthopaedic RCTs with negative or neutral findings at a rate that significantly differs from orthopaedic journals. However, the higher impact and digital coverage of the studies published in medical journals may disproportionally influence the practices of nonorthopaedic providers. Raising awareness of critical findings published in orthopaedic journals may be particularly important for improving healthcare policies and orthopaedic referral patterns for musculoskeletal problems.

High social media attention scores are not reflective of study quality: an altmetrics-based content analysis

Recent literature has demonstrated the associations between social media attention, as measured by altmetric attention score (AAS), and higher citation rates across medical disciplines. Despite increasing use of AAS, an understanding of factors associated with higher AAS and social media attention remains lacking. Furthermore, if this increased attention correlates with a higher methodological quality and lower biases has not been determined. Therefore, the purpose of the current study was to determine the relationship between methodological quality, study biases and the AAS in randomized controlled trials (RCTs). All RCTs from 2016 in the New England Journal of Medicine (NEJM), Journal of the American Medical Society (JAMA), and Lancet were extracted and the (1) AAS; (2) Methodological Bias (JADAD Scale); Study Bias (Cochrane Risk-of-Bias tool for RCTs) recorded. A total of 296 RCTs with a median (range) AAS and citation rate per article of 234.0(7-4079) and 165.0(4-3257), respectively, were included. The AAS was positively associated with citation rate (β 0.19, 95% CI 0.10-0.29; P < 0.001). Methodological bias was not associated with the AAS (β - 36.3, 95% CI - 83.5-10.9; P = 0.131), but was negatively associated with higher citation rates (β - 66.4, 95% CI - 106.0 to - 26.9; P = 0.001). The number of study biases was not associated with the AAS (β 43.7, 95% CI - 6.3-93.7;P = 0.086), but was positively associated with a higher citation rate (β 64.5, 95% CI 22.4-106.6; P = 0.003). The online attention of RCTs in medical journals was not necessarily reflective of high methodological quality and minimal study biases, but was associated with higher citation rates. Researchers and clinicians should critically examine each article despite the amount of online attention an article receives as the AAS does not necessarily reflect article quality.

Orthopaedic Randomized Controlled Trials Published in General Medical Journals Are Associated With Higher Altmetric Attention Scores and Social Media Attention Than Nonorthopaedic Randomized Controlled Trials

PURPOSE

To (1) compare the Altmetric Attention Score (AAS) and citation rates between orthopaedic and nonorthopaedic randomized controlled trials (RCTs) from 5 high-impact medical journals and (2) identify general characteristics of these articles associated with greater exposure on social media platforms.

METHODS

Articles published in The New England Journal of Medicine (NEJM), Lancet, The Journal of the American Medical Association (JAMA), Annals of Internal Medicine, and Archives of Internal Medicine between January 2011 and December 2016 were analyzed. These journals were selected based on retaining high impact factors with rigorous publication standards and availability of the AAS for their publications. The queried time frame was chosen to balance the inception of the AAS with an optimal period for citation accrual. A total of 14 article characteristics, in addition to number of Tweets, Facebook shares, news mentions, and the AAS, were extracted. Inclusion criteria were orthopaedic RCTs reporting on outcomes after surgical intervention. Linear regression was used to assess the relationship between publication characteristics and the AAS and social media attention.

RESULTS

A total of 9 orthopaedic and 59 nonorthopaedic RCTs were included. The mean AASs were significantly different (574 ± 565.7 versus 256.9 ± 222.3, P = .003), whereas citation rate was not (192.2 ± 117.1 versus 382.3 ± 560.3, P = .317). Orthopaedic RCTs had a significantly greater number of mentions on Twitter and Facebook (P < .001). A higher AAS significantly associated with a greater number of citations (β = 0.75, P = .019) for orthopaedic RCTs. The mean AAS of orthopaedic RCTs favoring nonoperative management (809.6 ± 676.3) was greater than those favoring operative treatment (292.0 ± 248.9) but was not statistically significant (P = .361).

CONCLUSION

Orthopaedic RCTs published in 5 high-impact general medical journals had a significantly greater mean AAS relative to nonorthopaedic RCTs, with no differences in citation rates. Additionally, there was a strong association between the AAS and citation rate of orthopaedic RCTs. Orthopaedic RCTs had greater social media exposure on both Twitter and Facebook.

CLINICAL RELEVANCE

Orthopaedic surgeons, researchers, and providers who publish RCTs in high-impact medical journals can anticipate extensive social media attention for their articles relative to other nonorthopaedic RCTs in the same journals. Social media attention may be related to operative versus nonoperative management topics. This study provides further evidence for the increasing use of the AAS and its association with citation accrual.

Sham Surgery Studies in Orthopaedic Surgery May Just Be a Sham: A Systematic Review of Randomized Placebo-Controlled Trials

PURPOSE

To determine the limitations of randomized sham surgery-controlled trials in orthopaedic sports medicine and fidelity of the trials' conclusions.

METHODS

Randomized placebo surgery-controlled trials in orthopaedic sports medicine were included in this Preferred Reporting Items for Systematic Reviews and Meta-Analyses-guided systematic review. Several aspects of investigation design and conduct were analyzed: genetic analysis for propensity to placebo response, equipoise of informed consent process, geography of trial subjects, percentage of eligible subjects willing to be randomized, changes from protocol publication to results publication, and perioperative complications.

RESULTS

Seven sham surgery-controlled trials (845 subjects [370 knees, 449 shoulders, 26 elbows]; 5 from Europe, 1 from North America, and 1 from Australia; all superiority model, efficacy design) were analyzed. There were consistent methodologic deficiencies across studies. No studies reported genetic analysis of susceptibility to placebo response. Three studies (43%) were underpowered. Crossover rates ranged from 8% to 36%, which led to un-blinding in up to 100% of subjects. There were low enrollment rates of eligible subjects (up to 57% refused randomization). Follow-up was short term (2 years or less in all but one study). Complication rates ranged from 0% to 12.5%, with complications occurring in both groups (no significant difference between groups in any study).

CONCLUSIONS

Randomized sham-controlled studies in orthopaedic sports medicine have significant methodologic deficiencies that may invalidate their conclusions. Randomized trial design (with or without placebo control) may be optimized through the inclusion of per-protocol analysis, blinding index, equivalence or noninferiority trial design, and a nontreatment group.

LEVEL OF EVIDENCE

Level II Systematic Review of Level II studies.

Trends in High-Impact Neurosurgical Randomized Controlled Trials Published in General Medical Journals: A Systematic Review

BACKGROUND

The neurosurgery literature lacks a comprehensive report of neurosurgical randomized controlled trials (RCTs) published in general medical journals. RCTs published in these journals have high visibility and impact on decision-making by general medical practitioners and health care policymakers.

METHODS

A systematic review of neurosurgical RCTs in the New England Journal of Medicine, The Lancet, Journal of the American Medical Association, The BMJ, and Annals of Internal Medicine was completed.

RESULTS

There were 78 neurosurgical RCTs published in the selected high-impact journals from 2000 to 2017. The most common study topics were neurovascular (n = 39, 50%) and spine (n = 24, 30.8%). Of these RCTs, 44 (56.4%) compared operative with nonoperative management. For studies published before 2017, the mean number of citations was 899. Approximately half of the studies showed superiority of operative management over nonoperative management in the intent to treat primary outcome of interest (n = 24, 54.5%). However, stratified by subsubspecialty, 7 (87.5%) of the functional RCTs, 9 (50%) of the neurovascular RCTs, 1 (50%) of the trauma RCTs, and 7 (43.8%) of the spinal RCTs demonstrated superiority of operative management over nonoperative management. Additionally, there were large subspecialty differences in study characteristics, such as rate of double blinding, proportion of patient enrollment from patients screened, and proportion of crossover from nonsurgical to surgical arm.

CONCLUSIONS

Neurosurgical RCTs in general medical journals have large subspecialty differences in characteristics such as crossovers from nonsurgical to surgical treatment arms and the proportion of studies demonstrating benefit of operative intervention over nonoperative management.

Trends in Foot and Ankle Studies Published in High-Impact General Medical Journals: A Systematic Review

The foot and ankle surgery literature lacks an analysis of studies published in general medical journals. Studies published in general medical journals have high visibility and significant impact on decision making by medical practitioners and health care policy makers. Knowledge of the subject matter and methodologic characteristics of foot and ankle publications being read by general medical practitioners and the lay public is important to the practicing foot and ankle surgeon. A systematic review of foot and ankle studies published from 2000 to 2017 in 5 high-impact general medical journals was undertaken. Data extracted included study topic, study design, study demographics, and methodologic characteristics of randomized controlled trials (RCTs). There were 47 foot and ankle studies published in the selected medical journals examined during the study time period. Most common topics were diabetic foot ulcers (n = 25 [53.2%]), plantar fasciitis (n = 6 [12.8%]), and Achilles tendinopathy (n = 5 [10.6%]). The most common study types were reviews (n = 24 [51.1%]), RCTs (n = 12 [25.4%], and prospective studies (n = 6 [12.8%]). Of the published RCTs, the most common journal of publication was the Journal of the American Medical Association (n = 7 [58.3%]). The median number of citations for RCTs published before 2016 was 242.5. Of the small number of RCTs that compared operative with nonoperative treatment, the majority supported nonoperative management (n = 3 [75%]). Diabetic foot ulcers, plantar fasciitis, and Achilles tendinopathy are the most commonly published foot and ankle topics in general medical journals.

Hip and Knee Arthroplasty Orthopedic Literature in Medical Journals-Is It Negatively Biased?

BACKGROUND

Healthcare policy is often determined by well-designed studies most often published in high-impact medical journals. However, concern about the presence of publication bias against lower-extremity arthroplasty-related studies has called into question some of the validity of certain reports. There are only a few studies investigating the presence of the bias in high-impact medical journals against lower-extremity arthroplasty intervention, particularly in the Journal of American Medical Association (JAMA), New England Journal of Medicine (NEJM), and the Lancet. Thus, the purpose of this study was to assess (1) the distribution of positive, neutral, and negative results; (2) the number of reports focused on lower-extremity arthroplasty complications among these 3 journals; and (3) difference in bias between 2 time periods (1975 to 1990 and 2000 to 2016).

METHODS

A review of the literature from 3 major medical journals (NEJM, Lancet, and JAMA) was performed using PubMed electronic databases, which retrieved articles between January 1976 and December 2016. Studies were categorized as being positive, neutral, and negatively biased by 2 reviewers. Studies were categorized as reporting on lower-extremity arthroplasty-related complications if they were based on complications including deep vein thrombosis, infection, metal-related complication, fat embolism, readmission, or mortality. In addition, we have compared the journal bias between 2 different time points (1975 to 1990 and 2000 to 2016). Descriptive analyses were performed to assess frequencies. Chi-squared analysis was conducted for categorical variables, whereas a z-test was performed for dichotomous data.

RESULTS

When assessing all 3 journals, there were 46 positive (30.3%), 46 negative (30.3%), and 60 neutral reports (39.5%). There was no statistically significant difference in classification proportions between the 3 groups (P = .905). There was a higher percentage of medical literature reporting on the complications of arthroplasty (55.9%); however, this was not statistically significant (z-score = 1.38; 95% confidence interval, 0.48-0.64; P = .17). There was no difference in overall journal reporting between 1975 to 1990 and 2000 to 2016 (P = .548).

CONCLUSION

There was no evidence of publication bias of lower-extremity arthroplasty reports in the major medical journals (JAMA, NEJM, and Lancet). However, there were more published studies reporting on complications of lower-extremity arthroplasty. This may be due to systematic bias among journal editors in these journals, or due to low journal submission reporting noncomplications after arthroplasty intervention. We did not find the time period to be a factor in bias reporting of orthopedic literature in major medical journals. More work is needed to verify the results of this study.