Nonscientific factors associated with acceptance for publication in The Journal of Bone and Joint Surgery (American Volume)

What is the sensitivity and specificity of the peer review process?

In this paper, we introduce the concepts of sensitivity and specificity to mathematically describe the accuracy of the peer review process. Sensitivity refers to the probability that the final decision for a manuscript would be acceptance, provided the manuscript meets the journal standards required for publication (i.e., true positive rate). Specificity refers to the probability that the final decision would be rejection, provided the work does not meet the standards required for publication (i.e., true negative rate). Therefore, in the peer review process, sensitivity measures the ability to correctly accept manuscripts that meet the required standards (true positives) and specificity measures the ability to correctly reject manuscripts that do not meet those quality standards required for publication (true negatives). Sensitivity and specificity values can inform the editor under what conditions the outcome of a peer review process becomes more precise and, therefore, if this does not occur, when the editor must improve the analysis involved in processing the information received from reviewers' reports. Sensitivity and specificity understood in this way can promote the ethical conduct of peer review processes and improve the validity of manuscript editorial decisions.

Comparison of the scientific performance in hip and knee arthroplasty between the leading continents

Background

Scientific progress in the field of knee and hip arthroplasty has enabled the preservation of mobility and quality of life in the case of patients with many primary degenerative and (post-) traumatic joint diseases. This comparative study aims to investigate differences in scientific performance between the leading continents in the field of hip and knee arthroplasty.

Methods

Using specific search terms all studies published by the scientific leading continents Europe, North America, Asia and Oceania listed in the Web of Science databases were included. All identified publications were analysed and comparative conclusions were drawn regarding the qualitative and quantitative scientific merit of each continent.

Results

Europe, followed by North America, Asia, and Oceania, had the highest overall number of publications in the field of arthroplasty. Since 2000, there has been a strong increase in knee arthroplasty publication rate, particular pronounced in Asia. Studies performed and published in North America and those on knee arthroplasty received the highest number of fundings. Publications regarding hip arthroplasty achieved the highest average citation rate. In contradistinction to the others, in North America most funding was provided by private agencies.

Conclusion

Although Europe showed the highest total number of publications, authors and institutions, arthroplasty research from North America received greater scientific attention and financial support. Measured by citations, publications on hip arthroplasty attained higher scientific interest and studies on knee arthroplasty received higher economic affection.

Middle East Authors' Contribution to the Journal of Arthroplasty’s Publications in the Past 20 years (2000–2020)

Background

Methods

Results

Conclusion

A multivariate model for successful publication of intensive care medicine randomized controlled trials in the highest impact factor journals: the SCOTI score

Background

Critical care randomized controlled trials (RCTs) are often published in high-impact journals, whether general journals [the New England Journal of Medicine (NEJM), The Lancet, the Journal of the American Medical Association (JAMA)] or critical care journals [Intensive Care Medicine (ICM), the American Journal of Respiratory and Critical Care Medicine (AJRCCM), Critical Care Medicine (CCM)]. As rejection occurs in up to 97% of cases, it might be appropriate to assess pre-submission probability of being published. The objective of this study was to develop and internally validate a simplified score predicting whether an ongoing trial stands a chance of being published in high-impact general journals.

Methods

A cohort of critical care RCTs published between 1999 and 2018 in the three highest impact medical journals (NEJM, The Lancet, JAMA) or the three highest impact critical care journals (ICM, AJRCCM, CCM) was split into two samples (derivation cohort, validation cohort) to develop and internally validate the simplified score. Primary outcome was journal of publication assessed as high-impact general journal (NEJM, The Lancet, JAMA) or critical care journal (ICM, AJRCCM, CCM).

Results

A total of 968 critical care RCTs were included in the predictive cohort and split into a derivation cohort (n = 510) and a validation cohort (n = 458). In the derivation cohort, the sample size (P value < 0.001), the number of centers involved (P value = 0.01), mortality as primary outcome (P value = 0.002) or a composite item including mortality as primary outcome (P value = 0.004), and topic [ventilation (P value < 0.001) or miscellaneous (P value < 0.001)] were independent factors predictive of publication in high-impact general journals, compared to high-impact critical care journals. The SCOTI score (Sample size, Centers, Outcome, Topic, and International score) was developed with an area under the ROC curve of 0.84 (95% Confidence Interval, 0.80–0.88) in validation by split sample.

Conclusions

The SCOTI score, developed and validated by split sample, accurately predicts the chances of a critical care RCT being published in high-impact general journals, compared to high-impact critical care journals.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13613-021-00954-x.

The Influence of Geographic Region on Hip and Knee Arthroplasty Literature From 1988 to 2018

Total joint arthroplasty constitutes a major focus of publications within orthopaedics. Because research expands and investigators from around the world contribute, it is important to understand the dynamics of publication.

Cost-effectiveness analyses in shoulder arthroplasty: a critical review using the Quality of Health Economic Studies (QHES) instrument

HYPOTHESIS

The purpose of this study was to perform a systematic review to identify cost-analysis studies pertaining to shoulder arthroplasty, provide a comprehensive review of published studies, and critically evaluate the quality of the available literature using the Quality of Health Economic Studies (QHES) instrument.

METHODS

A systematic review of the literature was performed to identify cost analyses examining shoulder arthroplasty. The inclusion criteria included studies pertaining to either shoulder hemiarthroplasty (HA), total shoulder arthroplasty (TSA), or reverse TSA. Articles were excluded based on the following: nonoperative studies, nonclinical studies, studies not based in the United States, and studies in which no cost analysis was performed. The quality of studies was assessed using the QHES instrument. One-sided Fisher exact testing was performed to identify predictors of both low-quality (ie, QHES score < 25th percentile) and high-quality (ie, QHES score > 75th percentile) cost analyses based on items within the QHES checklist.

RESULTS

Of the 196 studies screened, 9 were included. Seven studies conducted cost analyses comparing reverse TSA vs. arthroscopic rotator cuff repair, HA, or total hip arthroplasty, and 2 studies examined TSA vs. HA for primary glenohumeral arthritis. The average QHES score among all studies was 86.22 ± 13.39 points. Failure to include an annual cost discounting rate was associated with a low-quality QHES score (P = .03). In addition, including a discussion of the magnitude and direction of potential biases was associated with a high-quality score (P = .03).

CONCLUSIONS

Shoulder arthroplasty is a cost-effective procedure when used to treat a multitude of shoulder pathologies. The overall quality of cost analysis in shoulder arthroplasty is relatively good, with an average QHES score of 86.22 points. Studies failing to include an annual cost discounting rate are more likely to score below the 25th percentile, whereas those including a discussion of the magnitude and direction of potential biases are more likely to achieve a score in excess of the 75th percentile.

Assessment and Trends of the Methodological Quality of the Top 50 Most Cited Articles on Patellar Instability

Background

Studies with a low level of evidence (LOE) have dominated the top cited research in many areas of orthopaedics. The wide range of treatment options for patellar instability necessitates an investigation to determine the types of studies that drive clinical practice.

Purpose

To determine (1) the top 50 most cited articles on patellar instability and (2) the correlation between the number of citations and LOE or methodological quality.

Study Design

Cross-sectional study.

Methods

The Scopus and Web of Science databases were assessed to determine the top 50 most cited articles on patellar instability between 1985 and 2019. Bibliographic information, number of citations, and LOE were collected. Methodological quality was calculated using the Modified Coleman Methodology Score (MCMS) and the Methodological Index for Non-Randomized Studies (MINORS). Mean citations and mean citation density (citations per year) were correlated with LOE, MCMS, and MINORS scores.

Results

Most studies were cadaveric (n = 10; 20.0%), published in the American Journal of Sports Medicine (n = 13; 26.0%), published between 2000 and 2009 (n = 41; 82.0%), and conducted in the United States (n = 17; 34.0%). The mean number of citations and the citation density were 158.61 ± 59.53 (range, 95.5-400.5) and 12.74 ± 5.12, respectively. The mean MCMS and MINORS scores were 59.62 ± 12.58 and 16.24 ± 3.72, respectively. No correlation was seen between mean number of citations or citation density versus LOE. A significant difference was found in the mean LOE of articles published between 1990 and 1999 (5.0 ± 0) versus those published between 2000 and 2009 (3.12 ± 1.38; P = .03) and between 2010 and 2019 (3.00 ± 1.10; P = .01).

Conclusion

There was a shift in research from anatomy toward outcomes in patellar instability; however, these articles demonstrated low LOE and methodological quality. Higher quality studies are necessary to establish informed standards of management of patellar instability.

The prevalence and impact of gender bias and sexual discrimination in orthopaedics, and mitigating strategies

AIMS

Gender bias and sexual discrimination (GBSD) have been widely recognized across a range of fields and are now part of the wider social consciousness. Such conduct can occur in the medical workplace, with detrimental effects on recipients. The aim of this review was to identify the prevalence and impact of GBSD in orthopaedic surgery, and to investigate interventions countering such behaviours.

METHODS

A systematic review was conducted by searching Medline, EMCARE, CINAHL, PsycINFO, and the Cochrane Library Database in April 2020, and Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines to which we adhered. Original research papers pertaining to the prevalence and impact of GBSD, or mitigating strategies, within orthopaedics were included for review.

RESULTS

Of 570 papers, 27 were eligible for inclusion. These were published between 1998 and 2020. A narrative review was performed in light of the significant heterogeneity displayed by the eligible studies. A total of 13 papers discussed the prevalence of GBSD, while 13 related to the impact of these behaviours, and six discussed mitigating strategies. GBSD was found to be common in the orthopaedic workplace, with all sources showing women to be the subjects. The impact of this includes poor workforce representation, lower salaries, and less career success, including in academia, for women in orthopaedics. Mitigating strategies in the literature are focused on providing female role models, mentors, and educational interventions.

CONCLUSION

GBSD is common in orthopaedic surgery, with a substantial impact on sufferers. A small number of mitigating strategies have been tested but these are limited in their scope. As such, the orthopaedic community is obliged to participate in more thoughtful and proactive strategies that mitigate against GBSD, by improving female recruitment and retention within the specialty. Cite this article: Bone Joint J 2020;102-B(11):1446-1456.

The prevalence and impact of gender bias and sexual discrimination in orthopaedics, and mitigating strategies

AIMS

Gender bias and sexual discrimination (GBSD) have been widely recognized across a range of fields and are now part of the wider social consciousness. Such conduct can occur in the medical workplace, with detrimental effects on recipients. The aim of this review was to identify the prevalence and impact of GBSD in orthopaedic surgery, and to investigate interventions countering such behaviours.

METHODS

A systematic review was conducted by searching Medline, EMCARE, CINAHL, PsycINFO, and the Cochrane Library Database in April 2020, and Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines to which we adhered. Original research papers pertaining to the prevalence and impact of GBSD, or mitigating strategies, within orthopaedics were included for review.

RESULTS

Of 570 papers, 27 were eligible for inclusion. These were published between 1998 and 2020. A narrative review was performed in light of the significant heterogeneity displayed by the eligible studies. A total of 13 papers discussed the prevalence of GBSD, while 13 related to the impact of these behaviours, and six discussed mitigating strategies. GBSD was found to be common in the orthopaedic workplace, with all sources showing women to be the subjects. The impact of this includes poor workforce representation, lower salaries, and less career success, including in academia, for women in orthopaedics. Mitigating strategies in the literature are focused on providing female role models, mentors, and educational interventions.

CONCLUSION

GBSD is common in orthopaedic surgery, with a substantial impact on sufferers. A small number of mitigating strategies have been tested but these are limited in their scope. As such, the orthopaedic community is obliged to participate in more thoughtful and proactive strategies that mitigate against GBSD, by improving female recruitment and retention within the specialty.

International authorship and collaboration across bioRxiv preprints

Preprints are becoming well established in the life sciences, but relatively little is known about the demographics of the researchers who post preprints and those who do not, or about the collaborations between preprint authors. Here, based on an analysis of 67,885 preprints posted on bioRxiv, we find that some countries, notably the United States and the United Kingdom, are overrepresented on bioRxiv relative to their overall scientific output, while other countries (including China, Russia, and Turkey) show lower levels of bioRxiv adoption. We also describe a set of 'contributor countries' (including Uganda, Croatia and Thailand): researchers from these countries appear almost exclusively as non-senior authors on international collaborations. Lastly, we find multiple journals that publish a disproportionate number of preprints from some countries, a dynamic that almost always benefits manuscripts from the US.

Last five years audit of Journal of Clinical Orthopaedics and Trauma (JCOT) - A scientometric analysis

BACKGROUND

We have studied the growth of Journal of Clinical Orthopaedics and Trauma (JCOT), the official journal of the Delhi Orthopaedic Association indexed in PubMed, Scopus and Embase, along with an analysis of citation trends and levels of evidence of published articles in the last five years.

METHODS

SCOPUS and PubMed databases (2015-19) were used to extract publication-related data for the last five years. The official website of the journal was also used to retrieve issue-specific data. An audit of recent publications in the JCOT was conducted focusing on number of issues and publications(year wise), types of articles published, review speed, level of evidence of publications, types of publication(speciality wise), number of publications in PubMed and SCOPUS, number of citations(year wise), national and international authorship and the Cite Scores.

RESULTS

The submissions were increased by 2.8 times and the citations rocketed to 16.6 times in 2019, compared to 2015. A total of 890 papers were there and the journal was cited 1702 times in the last five years. More articles with higher levels of evidence papers are now being published.

CONCLUSION

There has been a significant increase in the numbers of submissions, publications, citations, Cite Score, h-index and contribution by the International authors. Its Editorial efficiency has also been outstanding with timely reviews and editorial decisions. The JCOT has become increasingly internationally relevant in the last five years, in the orthopaedic community. However, JCOT needs to publish more papers with a higher level of evidence like Systematic reviews, Meta-analyses, Randomized Controlled Trials and should only publish top-class lower hierarchy papers (like Case reports, Case-controlled and Retrospective studies).

Industry sponsorship and research outcome

BACKGROUND

Clinical research affecting how doctors practice medicine is increasingly sponsored by companies that make drugs and medical devices. Previous systematic reviews have found that pharmaceutical-industry sponsored studies are more often favorable to the sponsor's product compared with studies with other sources of sponsorship. A similar association between sponsorship and outcomes have been found for device studies, but the body of evidence is not as strong as for sponsorship of drug studies. This review is an update of a previous Cochrane review and includes empirical studies on the association between sponsorship and research outcome.

OBJECTIVES

To investigate whether industry sponsored drug and device studies have more favorable outcomes and differ in risk of bias, compared with studies having other sources of sponsorship.

SEARCH METHODS

In this update we searched MEDLINE (2010 to February 2015), Embase (2010 to February 2015), the Cochrane Methodology Register (2015, Issue 2) and Web of Science (June 2015). In addition, we searched reference lists of included papers, previous systematic reviews and author files.

SELECTION CRITERIA

Cross-sectional studies, cohort studies, systematic reviews and meta-analyses that quantitatively compared primary research studies of drugs or medical devices sponsored by industry with studies with other sources of sponsorship. We had no language restrictions.

DATA COLLECTION AND ANALYSIS

Two assessors screened abstracts and identified and included relevant papers. Two assessors extracted data, and we contacted authors of included papers for additional unpublished data. Outcomes included favorable results, favorable conclusions, effect size, risk of bias and whether the conclusions agreed with the study results. Two assessors assessed risk of bias of included papers. We calculated pooled risk ratios (RR) for dichotomous data (with 95% confidence intervals (CIs)).

MAIN RESULTS

Twenty-seven new papers were included in this update and in total the review contains 75 included papers. Industry sponsored studies more often had favorable efficacy results, RR: 1.27 (95% CI: 1.17 to 1.37) (25 papers) (moderate quality evidence), similar harms results RR: 1.37 (95% CI: 0.64 to 2.93) (four papers) (very low quality evidence) and more often favorable conclusions RR: 1.34 (95% CI: 1.19 to 1.51) (29 papers) (low quality evidence) compared with non-industry sponsored studies. Nineteen papers reported on sponsorship and efficacy effect size, but could not be pooled due to differences in their reporting of data and the results were heterogeneous. We did not find a difference between drug and device studies in the association between sponsorship and conclusions (test for interaction, P = 0.98) (four papers). Comparing industry and non-industry sponsored studies, we did not find a difference in risk of bias from sequence generation, allocation concealment, follow-up and selective outcome reporting. However, industry sponsored studies more often had low risk of bias from blinding, RR: 1.25 (95% CI: 1.05 to 1.50) (13 papers), compared with non-industry sponsored studies. In industry sponsored studies, there was less agreement between the results and the conclusions than in non-industry sponsored studies, RR: 0.83 (95% CI: 0.70 to 0.98) (six papers).

AUTHORS' CONCLUSIONS

Sponsorship of drug and device studies by the manufacturing company leads to more favorable efficacy results and conclusions than sponsorship by other sources. Our analyses suggest the existence of an industry bias that cannot be explained by standard 'Risk of bias' assessments.

The Early Impact of an Administrative Processing Fee on Manuscript Submissions at The Journal of Bone & Joint Surgery

BACKGROUND

There was a dramatic increase in the volume of manuscripts submitted to The Journal of Bone & Joint Surgery (JBJS) between 2009 and 2012. This resulted in increased journal administrative costs. To offset this financial burden, in May 2013, JBJS started charging authors an administrative processing fee at the time of submission. The purpose of this study was to assess the impact of the administrative fee on the volume and characteristics of manuscripts submitted to JBJS.

METHODS

Our analysis included 866 manuscripts submitted to JBJS between November 2012 and November 2013. We compared manuscripts submitted 6 months prior to fee implementation and prior to the announcement (denoted as the baseline group), in the several months prior to fee implementation but after the fee implementation announcement (denoted as the fee announcement group), and in the 6 months after fee implementation (denoted as the fee implementation group). Manuscripts were reviewed for institutional and author demographic characteristics, as well as for general study characteristics.

RESULTS

In the first full calendar year (2014) after the implementation of the fee, the annual volume of submissions to JBJS declined by 33.5% compared with the annual submission volume in 2010 to 2012. In a comparative analysis, the geographical region of origin (p = 0.003), level of evidence (p < 0.0001), funding, and specialty differed between the 3 submission periods. However, subgroup analyses demonstrated that differences were attributable to the fee announcement group and that there were few important differences between the baseline and fee implementation groups. Reporting of funding information improved significantly between the baseline and fee implementation groups; in the post-fee implementation period, studies were more likely to have declared no external funding source (p = 0.001).

CONCLUSIONS

The administrative processing fee at JBJS has been associated with a decrease in submission volume, but, overall, there has not been a change in the characteristics of studies submitted. However, decreased overall volume implies a decrease in the absolute number of high-level studies submitted to the journal. Administrative processing fees at high-volume journals may be a financially viable way to offset high administrative costs without substantially changing the characteristics of submitted articles.

Complications and Adverse Events of a Randomized Clinical Trial Comparing 3 Graft Types for ACL Reconstruction

OBJECTIVE

Complications/adverse events of anterior cruciate ligament (ACL) surgery are underreported, despite pooled level 1 data in systematic reviews. All adverse events/complications occurring within a 2-year postoperative period after primary ACL reconstruction, as part of a large randomized clinical trial (RCT), were identified and described.

DESIGN

Prospective, double-blind randomized clinical trial. Patients and the independent trained examiner were blinded to treatment allocation.

SETTING

University-based orthopedic referral practice.

PATIENTS

Three hundred thirty patients (14-50 years; 183 males) with isolated ACL deficiency were intraoperatively randomized to ACL reconstruction with 1 autograft type. Graft harvest and arthroscopic portal incisions were identical.

INTERVENTION

Patients were equally distributed to patellar tendon (PT), quadruple-stranded hamstring tendon (HT), and double-bundle (DB) hamstring autograft ACL reconstruction.

MAIN OUTCOME MEASURES

Adverse events/complications were patient reported, documented, and diagnoses confirmed.

RESULTS

Two major complications occurred: pulmonary embolism and septic arthritis. Twenty-four patients (7.3%) required repeat surgery, including 25 separate operations: PT = 7 (6.4%), HT = 9 (8.2%), and DB = 8 (7.3%). Repeat surgery was performed for meniscal tears (3.6%; n = 12), intra-articular scarring (2.7%; n = 9), chondral pathology (0.6%; n = 2), and wound dehiscence (0.3%; n = 1). Other complications included wound problems, sensory nerve damage, muscle tendon injury, tibial periostitis, and suspected meniscal tears and chondral lesions. Overall, more complications occurred in the HT/DB groups (PT = 24; HT = 31; DB = 45), but more PT patients complained of moderate or severe kneeling pain (PT = 17; HT = 9; DB = 4) at 2 years.

CONCLUSIONS

Overall, ACL reconstructive surgery is safe. Major complications were uncommon. Secondary surgery was necessary 7.3% of the time for complications/adverse events (excluding graft reinjury or revisions) within the first 2 years.

LEVEL OF EVIDENCE

Level 1 (therapeutic studies).

CLINICAL RELEVANCE

This article reports on the complications/adverse events that were prospectively identified up to 2 years postoperatively, in a defined patient population participating in a large double-blind randomized clinical trial comparing PT, single-bundle hamstring, and DB hamstring reconstructions for ACL rupture.

The Quality of Cost-Utility Analyses in Orthopedic Trauma

As health care in the United States transitions toward a value-based model, there is increasing interest in applying cost-effectiveness analysis within orthopedic surgery. Orthopedic trauma care has traditionally underemphasized economic analysis. The goals of this review were to identify US-based cost-utility analysis in orthopedic trauma, to assess the quality of the available evidence, and to identify cost-effective strategies within orthopedic trauma. Based on a review of 971 abstracts, 8 US-based cost-utility analyses evaluating operative strategies in orthopedic trauma were identified. Study findings were recorded, and the Quality of Health Economic Studies (QHES) instrument was used to grade the overall quality. Of the 8 studies included in this review, 4 studies evaluated hip and femur fractures, 3 studies analyzed upper extremity fractures, and 1 study assessed open tibial fracture management. Cost-effective interventions identified in this review include total hip arthroplasty (over hemiarthroplasty) for femoral neck fractures in the active elderly, open reduction and internal fixation (over nonoperative management) for distal radius and scaphoid fractures, limb salvage (over amputation) for complex open tibial fractures, and systems-based interventions to prevent delay in hip fracture surgery. The mean QHES score of the studies was 79.25 (range, 67-89). Overall, there is a paucity of cost-utility analyses in orthopedic trauma; however, the available evidence suggests that certain operative interventions can be cost-effective. The quality of these studies, however, is fair, based on QHES grading. More attention should be paid to evaluating the cost-effectiveness of operative intervention in orthopedic trauma.

Cost-effectiveness analyses in orthopaedic sports medicine: a systematic review

BACKGROUND

As increasing attention is paid to the cost of health care delivered in the United States (US), cost-effectiveness analyses (CEAs) are gaining in popularity. Reviews of the CEA literature have been performed in other areas of medicine, including some subspecialties within orthopaedics. Demonstrating the value of medical procedures is of utmost importance, yet very little is known about the overall quality and findings of CEAs in sports medicine.

PURPOSE

To identify and summarize CEA studies in orthopaedic sports medicine and to grade the quality of the available literature.

STUDY DESIGN

Systematic review.

METHODS

A systematic review of the literature was performed to compile findings and grade the methodological quality of US-based CEA studies in sports medicine. The Quality of Health Economic Studies (QHES) instrument and the checklist by the US Panel on Cost-effectiveness in Health and Medicine were used to assess study quality. One-sided Fisher exact testing was performed to analyze the predictors of high-quality CEAs.

RESULTS

Twelve studies met inclusion criteria. Five studies examined anterior cruciate ligament reconstruction, 3 studies examined rotator cuff repair, 2 examined autologous chondrocyte implantation, 1 study examined hip arthroscopic surgery, and 1 study examined the operative management of shoulder dislocations. Based on study findings, operative intervention in sports medicine is highly cost-effective. The quality of published evidence is good, with a mean quality score of 81.8 (range, 70-94). There is a trend toward higher quality in more recent publications. No significant predictor of high-quality evidence was found.

CONCLUSION

The CEA literature in sports medicine is good; however, there is a paucity of studies, and the available evidence is focused on a few procedures. More work needs to be conducted to quantify the cost-effectiveness of different techniques and procedures within sports medicine. The QHES tool may be useful for the evaluation of future CEAs.

Current status of cost utility analyses in total joint arthroplasty: a systematic review

BACKGROUND

Total joint arthroplasty (TJA), although considered to be highly beneficial, is associated with substantial costs to the US healthcare system. Cost utility analysis has become an increasingly important means to objectively evaluate the value of a healthcare intervention from the perspective of both extending the quantity and improving the quality of life. Relatively little is known about the overall cost utility analysis evidence base in TJA.

QUESTIONS/PURPOSES

The goals of this review were to (1) determine the cost utility of TJA interventions; (2) critically assess the quality of published US-based cost utility analyses using the Quality of Health Economic Studies instrument; and (3) determine what characteristics were common among studies receiving a high quality score.

METHODS

A systematic review of the literature using the MEDLINE database was performed to compile findings and critically appraise US-based cost utility analysis studies for total hip and knee arthroplasty. Based on review of 676 identified articles, 23 studies were included. We used the Quality of Health Economic Studies instrument to assess study quality and one-sided Fisher's exact tests were applied to analyze the predictors of high-quality cost utility analysis.

RESULTS

Very few studies compare the cost utility of TJA versus nonoperative intervention; however, the available evidence suggests that TJA can be cost-saving and is highly cost-effective compared with conservative management of end-stage arthritis. The majority of identified studies are focused on the cost utility of new implant technologies or comparisons among surgical alternatives. These studies suggest that the upfront costs associated with new technologies are cost-effective when there is a major reduction in a future cost. The quality of identified studies is quite high (Quality of Health Economic Studies Instrument score: mean 86.5; range, 63-100). National funding source (p = 0.095) and lifetime horizon for analysis (p = 0.07) correlate with high-quality evidence but do not reach significance.

CONCLUSIONS

Over the past 15 years, there has been a major increase in the volume of cost utility analyses published in total hip and knee arthroplasty. The quality of cost utility analyses published during that period is good. As increasing attention is paid to value in US health care, more attention should be paid to understanding the cost utility of TJA compared with nonoperative treatment modalities. Future studies may also look to incorporate patient willingness to pay.

Cost-utility analyses in spine care: a qualitative and systematic review

STUDY DESIGN

Systematic review.

OBJECTIVE

A systematic review was performed to identify US-based cost-utility analyses (CUA) studies in spine care and to critically evaluate the quality of the available literature.

SUMMARY OF BACKGROUND DATA

There has been a recent trend in the United States toward increased publication of economic analyses in spine care. The cost-effectiveness of spine interventions and the quality of published literature is not well understood.

METHODS

A MEDLINE search was conducted to identify cost analyses in spine care. Articles were excluded on the basis of the following criteria: nonspine care, nonoperative, non-US based, nonclinical, and not CUA. Of the 424 screened articles, 20 met inclusion criteria. Quality of studies was assessed using the Quality of Health Economic Studies instrument.

RESULTS

Evidence for the cost-effectiveness of operative spinal intervention is varied. The majority of available studies report favorable cost-effectiveness ratios, however, a few studies suggest that certain operative interventions are not cost-effective. Average Quality of Health Economic Studies score of all included studies was 75.1 (60-93). The quality of evidence is variable and there are a number of weaknesses in the available literature, most significant of which is that few studies adopt a long-term time horizon or have sufficient follow-up (N = 3/20). High Quality of Health Economic Studies scoring studies were more likely to have sensitivity analysis (P = 0.016), societal cost perspective (P = 0.014), and a funding disclosure (P = 0.03).

CONCLUSION

There is a small but rapidly growing body of US-based CUA in spine care. The quality of CUA evidence is variable but there are significant opportunities to strengthen future CUA studies in spine. This study highlights the need for more attention to CUA research and the quality of these studies in spine care.

Role of editorial and peer review processes in publication bias: analysis of drug trials submitted to eight medical journals

BACKGROUND

Publication bias is generally ascribed to authors and sponsors failing to submit studies with negative results, but may also occur after submission. We evaluated whether submitted manuscripts on randomized controlled trials (RCTs) with drugs are more likely to be accepted if they report positive results.

METHODS

Manuscripts submitted from January 2010 through April 2012 to one general medical journal (BMJ) and seven specialty journals (Annals of the Rheumatic Diseases, British Journal of Ophthalmology, Gut, Heart, Thorax, Diabetologia, and Journal of Hepatology) were included, if at least one study arm assessed the efficacy or safety of a drug and a statistical test was used to evaluate treatment effects. Publication status was retrospectively retrieved from submission systems or provided by journals. Sponsorship and trial results were extracted from manuscripts and classified according to predefined criteria. Main outcome measure was acceptance for publication.

RESULTS

Of 15,972 manuscripts submitted, 472 (3.0%) were drug RCTs, of which 98 (20.8%) were published. Among submitted drug RCTs, 287 (60.8%) had positive and 185 (39.2%) negative results. Of these, 60 (20.9%) and 38 (20.5%), respectively, were published. Manuscripts on non-industry trials (n = 213) reported positive results in 138 (64.8%) manuscripts, compared to 71 (47.7%) on industry-supported trials (n = 149), and 78 (70.9%) on industry-sponsored trials (n = 110). Twenty-seven (12.7%) non-industry trials were published, compared to 27 (18.1%) industry-supported and 44 (40.0%) industry-sponsored trials. After adjustment for other trial characteristics, manuscripts reporting positive results were not more likely to be published (OR, 1.00; 95% CI, 0.61 to 1.66). Submission to specialty journals, sample size, multicentre status, journal impact factor, and corresponding authors from Europe or US were significantly associated with publication.

CONCLUSIONS

For the selected journals, there was no tendency to preferably publish manuscripts on drug RCTs that reported positive results, suggesting that publication bias may occur mainly prior to submission.

The reporting of functional outcome instruments in the Journal of Orthopaedic Trauma over a 5-year period

OBJECTIVES

Orthopaedic journals, such as the Journal of Orthopaedic Trauma, frequently publish studies reporting functional outcome instruments, but little information has been provided regarding the validity and overall strength of these instruments. This study analyzes the trends in reported functional outcome instruments in articles published in the Journal of Orthopaedic Trauma over a 5-year period and examines the utilization rate, "overall" strength, and validity of these functional outcome instruments for the populations being studied.

METHODS

Articles that were published in the Journal of Orthopaedic Trauma from January 2006 to December 2010 were reviewed, and each article was assigned to 1 of 4 different categories, based on the subspecialty focus and body region. The total number of articles reporting the use of functional outcome instruments, articles with at least 1 functional outcome instrument found in the AO Handbook, and the total number of functional outcome instruments reported were recorded. Each functional outcome instrument was assigned to 1 of 3 categories (generic, nonvalidated, validated), and each validated instrument was also examined to determine whether the category of interest for which it was used was one in which it was previously validated in.

RESULTS

A total of 171 articles (34%) of the articles initially reviewed met the inclusion criteria. The average number of articles per year that reported functional outcome instruments was 56% (range, 47%-65%), and the average number of articles that reported at least 1 validated outcome instrument was 51% (range, 44%-61%). The average percentage of validated scores that were appropriately used within the category of interest was 23% (range, 13%-41%).

CONCLUSIONS

Even though the 56% utilization rate of functional outcome instruments in The Journal of Orthopaedic Trauma is much higher than other journals, it is still low given the importance of measuring and attaining excellent functional outcomes. It is clear that future effort should be given to validating outcome measures for correct evaluation of orthopaedic trauma patients.

Demographics of disclosure of conflicts of interest at the 2011 annual meeting of the American Academy of Orthopaedic Surgeons

BACKGROUND

There is growing concern regarding conflicts of interest in orthopaedic research and education. Because of their potential influence on orthopaedic practice, conflicts of interest among participants in the educational programs of the American Academy of Orthopaedic Surgeons (AAOS) are of particular interest.

METHODS

We analyzed the voluntarily disclosed conflicts of interest listed in the Final Program of the 2011 Annual Meeting of the AAOS for the relevant program committees as well as for presentations in the disciplines of pediatric orthopaedic surgery, spine, and sports medicine/arthroscopy.

RESULTS

Conflicts of interest were disclosed by participants for each of the program committees and for over 75% of the presentations. Conflicts of interest were disclosed for 100% of the featured symposia, 80% of the scientific exhibits, 76% of the podium presentations, and 75% of the posters. Over half (53%) of the disclosures were for paid consultancy, 51% were for research support for the principal investigator, 41% were for paid presentations, 39% were for royalties, and 39% were for stock. The highest number of disclosures for an individual author was thirty-seven. The number of disclosures per author was significantly (p &lt; 0.001) correlated to the number of presentations per author. Disclosures were associated with 379 different companies; relationships with a relatively small number (twenty-six, 7%) of these companies were listed in the disclosures for 67% of the presentations.

CONCLUSIONS

Voluntarily disclosed conflicts of interest were common at the 2011 AAOS Annual Meeting, especially for the featured symposia. In view of the previously documented frequency of undisclosed conflicts of interest, as well as the previously documented effects of conflicts of interest on research design, conduct, and conclusion, it may be time to consider improved strategies for ensuring the accuracy and completeness of disclosure and for managing the biasing effects of conflicts of interest.

Industry sponsorship and research outcome

BACKGROUND

Clinical research affecting how doctors practice medicine is increasingly sponsored by companies that make drugs and medical devices. Previous systematic reviews have found that pharmaceutical industry sponsored studies are more often favorable to the sponsor's product compared with studies with other sources of sponsorship. This review is an update using more stringent methodology and also investigating sponsorship of device studies.

OBJECTIVES

To investigate whether industry sponsored drug and device studies have more favorable outcomes and differ in risk of bias, compared with studies having other sources of sponsorship.

SEARCH METHODS

We searched MEDLINE (1948 to September 2010), EMBASE (1980 to September 2010), the Cochrane Methodology Register (Issue 4, 2010) and Web of Science (August 2011). In addition, we searched reference lists of included papers, previous systematic reviews and author files.

SELECTION CRITERIA

Cross-sectional studies, cohort studies, systematic reviews and meta-analyses that quantitatively compared primary research studies of drugs or medical devices sponsored by industry with studies with other sources of sponsorship. We had no language restrictions.

DATA COLLECTION AND ANALYSIS

Two assessors identified potentially relevant papers, and a decision about final inclusion was made by all authors. Two assessors extracted data, and we contacted authors of included papers for additional unpublished data. Outcomes included favorable results, favorable conclusions, effect size, risk of bias and whether the conclusions agreed with the study results. Two assessors assessed risk of bias of included papers. We calculated pooled risk ratios (RR) for dichotomous data (with 95% confidence intervals).

MAIN RESULTS

Forty-eight papers were included. Industry sponsored studies more often had favorable efficacy results, risk ratio (RR): 1.24 (95% confidence interval (CI): 1.14 to 1.35), harms results RR: 1.87 (95% CI: 1.54 to 2.27) and conclusions RR: 1.31 (95% CI: 1.20 to 1.44) compared with non-industry sponsored studies. Ten papers reported on sponsorship and effect size, but could not be pooled due to differences in their reporting of data. The results were heterogeneous; five papers found larger effect sizes in industry sponsored studies compared with non-industry sponsored studies and five papers did not find a difference in effect size. Only two papers (including 120 device studies) reported separate data for devices and we did not find a difference between drug and device studies on the association between sponsorship and conclusions (test for interaction, P = 0.23). Comparing industry and non-industry sponsored studies, we did not find a difference in risk of bias from sequence generation, allocation concealment and follow-up. However, industry sponsored studies more often had low risk of bias from blinding, RR: 1.32 (95% CI: 1.05 to 1.65), compared with non-industry sponsored studies. In industry sponsored studies, there was less agreement between the results and the conclusions than in non-industry sponsored studies, RR: 0.84 (95% CI: 0.70 to 1.01).

AUTHORS' CONCLUSIONS

Sponsorship of drug and device studies by the manufacturing company leads to more favorable results and conclusions than sponsorship by other sources. Our analyses suggest the existence of an industry bias that cannot be explained by standard 'Risk of bias' assessments.

The fate of manuscripts rejected by The Journal of Bone and Joint Surgery (American Volume)

BACKGROUND

Of the many manuscripts that are submitted to The Journal of Bone and Joint Surgery (American Volume) (JBJS-A) for publication, the majority are not accepted. However, little is known about the outcome of these rejected submissions. To determine the fate of studies rejected by JBJS-A, we conducted a follow-up investigation of all clinical and basic science manuscripts that were submitted to The Journal between January 2004 and June 2005 but were not accepted.

METHODS

For each rejected manuscript, data were extracted on a wide variety of scientific and nonscientific characteristics, which were plausibly related to subsequent publication. PubMed searches were conducted to determine which manuscripts achieved full publication within five years, and logistic regression was used to identify the factors associated with publication. To further elucidate the factors associated with publication, a survey was administered to the corresponding author of each rejected manuscript.

RESULTS

At five years following rejection by JBJS-A, 75.8% (696 of 918) of manuscripts had reached full publication. In the multivariate analysis, factors associated with a higher likelihood of subsequent publication included grade of initial review by JBJS-A (p = 0.029), disclosure of a for-profit or nonprofit conflict of interest (p = 0.028 and 0.027, respectively), and a greater number of prior publications in frequently cited orthopaedic journals by the corresponding author (p < 0.0001). Manuscripts were less likely to reach full publication if the corresponding author was from Asia or the Middle East (p = 0.004) or was a woman (p = 0.003). Among survey respondents who indicated that their study had not yet reached full publication, the most commonly cited reason was lack of time (reported by 51.4% of respondents [thirty-eight of seventy-four]).

CONCLUSIONS

Most manuscripts (75.8%) not accepted by JBJS-A were published elsewhere within five years of rejection. The factors predictive of subsequent publication were primarily investigator-related as opposed to study-related. Given this low threshold for eventual publication, readers are encouraged to use criteria other than inclusion in the PubMed database to identify high-quality papers.

Self-reported financial conflicts of interest during scientific presentations in emergency medicine

OBJECTIVES

This study was a review of the scientific abstracts presented at a national conference for the required conflict of interest (COI) disclosure both before the meeting and during presentation.

METHODS

All presenters were given specific instructions regarding COI reporting at the time of abstract acceptance. All poster presentations were required to have a COI statement. Three physicians using standardized data abstraction forms reviewed abstracts accepted for poster presentation at the 2010 annual meeting of the Society for Academic Emergency Medicine (SAEM). Posters were reviewed for the presence of a required COI disclosure statement, and these results were compared to the mandatory continuing medical education (CME) disclosure form that was sent by the presenters to the SAEM central office before the meeting.

RESULTS

There were 412 posters accepted for presentation at the 2010 SAEM annual meeting. The reviewers observed 382 (93%) of the total posters for the conference. Sixty-nine abstracts (18%) reported a COI. Only 26 (38%) of these were actually reported to the SAEM office on the CME disclosure form before the meeting; the remaining 62% were found on the poster alone. COI that were reported on the CME disclosure form were found on the poster 46% of the time. The remaining posters without a COI actually displayed the mandatory disclosure statement only 14% of the time.

CONCLUSIONS

This review of presentations at a national meeting found a lack of compliance with printed guidelines for COI disclosure during scientific presentation. Efforts to increase uniformity and clarity may result in increased compliance.

A survey of orthopaedic journal editors determining the criteria of manuscript selection for publication

BACKGROUND

To investigate the characteristics of editors and criteria used by orthopaedic journal editors in assessing submitted manuscripts.

METHODS

Between 2008 to 2009 all 70 editors of Medline listed orthopaedic journals were approached prospectively with a questionnaire to determine the criteria used in assessing manuscripts for publication.

RESULTS

There was a 42% response rate. There was 1 female editor and the rest were male with 57% greater than 60 years of age. 67% of the editors worked in university teaching hospitals and 90% of publications were in English. The review process differed between journals with 59% using a review proforma, 52% reviewing an anonymised manuscript, 76% using a routine statistical review and 59% of journals used 2 reviewers routinely. In 89% of the editors surveyed, the editor was able to overrule the final decision of the reviewers. Important design factors considered for manuscript acceptance were that the study conclusions were justified (80%), that the statistical analysis was appropriate (76%), that the findings could change practice (72%). The level of evidence (70%) and type of study (62%) were deemed less important. When asked what factors were important in the manuscript influencing acceptance, 73% cited an understandable manuscript, 53% cited a well written manuscript and 50% a thorough literature review as very important factors.

CONCLUSIONS

The editorial and review process in orthopaedic journals uses different approaches. There may be a risk of language bias among editors of orthopaedic journals with under-representation of non-English publications in the orthopaedic literature.

Editorial bias in scientific publications

INTRODUCTION

Many authors believe that there are biases in scientific publications. Editorial biases include publication bias; which refers to those situations where the results influence the editor's decision, and editorial bias refers to those situations where factors related with authors or their environment influence the decision.

DEVELOPMENT

This paper includes an analysis of the situation of editorial biases. One bias is where mainly articles with positive results are accepted, as opposed to those with negative results. Another is latent bias, where positive results are published before those with negative results. In order to examine editorial bias, this paper analyses the influence of where the article originated; the country or continent, academic centre of origin, belonging to cooperative groups, and the maternal language of the authors. The article analyses biases in the editorial process in the publication of funded clinical trials.

CONCLUSIONS

Editorial biases exists. Authors, when submitting their manuscript, should analyse different journals and decide where their article will receive adequate treatment.

Relationship between declared funding support and level of evidence

BACKGROUND

The relationship between industry and the orthopaedic community is under increasing scrutiny. Industry traditionally has funded a substantial amount of the orthopaedic research published in this and other journals. The objective of the present study was to investigate associations between the level of evidence and declared source(s) of funding in papers published in the American volume of The Journal of Bone and Joint Surgery.

METHODS

All articles published in the American volume of The Journal of Bone and Joint Surgery from January 2003 to December 2007 were reviewed by a single individual. Primary research papers with an assigned level of evidence were assessed with regard to source of funding, subject area, and results. The association between source of funding and level of evidence was described with use of contingency tables and chi-square tests.

RESULTS

Of 886 studies with an assigned level of evidence, 246 were funded by industry, of which 124 (50%) were graded as Level-IV evidence. Among 274 studies funded by governments, foundations, or universities, only seventy-nine (29%) were graded as Level-IV evidence. Among 366 studies with no funding declared, 209 (57%) were graded as Level-IV evidence. The association between industry funding and a lower level of evidence was significant (p < 0.0001).

CONCLUSIONS

While industry funded a larger number of studies than any other single source in this journal, the level of evidence of industry-funded studies was lower that that for studies funded by governments, foundations, or universities. Improving the scientific quality of industry-funded research might increase the quality of evidence for making orthopaedic decisions.

Publication bias in Kienböck's disease: systematic review

PURPOSE

Kienböck's disease is considered rare and currently affects fewer than 200,000 people in the United States. Given the inherent challenges associated with researching rare diseases, the intense effort in hand surgery to treat this uncommon disorder may be influenced by publication bias in which positive outcomes are preferentially published. The specific aim of this project was to conduct a systematic review of the literature with the hypothesis that publication bias is present for the treatment of Kienböck's disease.

METHODS

We conducted a systematic review of all available abstracts associated with published manuscripts (English and non-English) and abstracts accepted to the 1992 to 2004 American Society for Surgery of the Hand (ASSH) annual meetings. Data collection included various study characteristics, direction of outcome (positive, neutral/negative), complication rates, mean follow-up time, time to publication, and length of patient enrollment.

RESULTS

Our study included 175 (124 English, 51 non-English) published manuscripts and 14 abstracts from the 1992 to 2004 annual ASSH meetings. Abstracts from published manuscripts were associated with a 53% positive outcome rate, which is lower than the 74% positive outcome rate found among other surgically treated disorders. Over the past 40 years, studies have become more positive (36% to 68%, p=.007) and are more likely to incorporate statistical analysis testing (0% to 55%, p<.001). Of the 14 abstracts accepted to ASSH, 11 were published in peer-reviewed journals. Ten of the 14 accepted abstracts were considered positive, and there was no significant difference in publication rate between studies with positive (n = 10) and negative (n = 4) outcomes (p>.999).

CONCLUSIONS

The acceptance rate for negative outcomes studies regarding Kienböck's disease is higher than for other surgical disorders. This may indicate a relative decrease in positive outcome bias among published Kienböck's disease studies compared with other surgical disorders. However, the increasing positive outcome rate for published Kienböck's disease studies over time may suggest a trend of increasing publication bias among journals toward Kienböck's disease studies.