Level of Clinical Evidence Presented at the Arthroscopy Association of North America Annual Meeting Over 10 Years (2006-2015)

Study Characteristics and Impact of Pediatric Orthopaedic Society of North America Annual Meeting's "Best Papers" From 2009 to 2019

OBJECTIVE

Award-winning abstracts are selected every year at the Pediatric Orthopaedic Society of North America (POSNA) annual meeting as "best paper" or poster. It is unknown how many achieve publication in peer-reviewed journals and the impact they have. We sought to determine the characteristics, including the level of evidence (LoE), publication rates, and the impact of award-winning abstracts on pediatric orthopaedic surgery practice from 2009 to 2019.

METHODS

Award-winning abstracts or "best papers" from 2009 to 2019 were retrospectively reviewed from the POSNA website from abstract publication to manuscript publication. A search across Pubmed was used to match abstracts to their publications by comparing author names, titles, study design and methodology, results, and conclusions. Area of focus, abstract authors, institutions, publication status, LoE, time to publication, authors of publication, journals of publication, and the journal's latest Hirsch Index and impact factor were recorded.

RESULTS

There have been 54 "best papers" at the POSNA annual meeting from 2009 to 2019. Of those, 39 have been published across 17 different journals for a publication rate of 72%. The average time from abstract presentation to publication was 21.2 months with a range of 0 to 121 months. Of the published award-winning abstracts, 64% (25) were published within 2 years, 87% (34) within 3 years, and 95% (37) within 4 years. Out of the published abstracts, 26% (10) were in the Journal of Pediatric Orthopaedics , 23% (9) were in the Journal of Bone and Joint Surgery , and 10% (4) were in the Journal of Child Orthopaedics . The median number of abstract authors was 4 and increased to a median of 6 authors once published. Most award-winning abstracts had a LoE of 3. The average journal impact factor for all publications was 4; the average Hirsch Index for the corresponding author was 29.9, and the average number of citations for a publication was 41 with a range of 0 to 270.

CONCLUSIONS

The majority of the "best papers" presented at POSNA annual meetings from 2009 to 2019 were published in peer-reviewed journals within 2 years of presentation, with approximately half being published in the Journal of Pediatric Orthopaedics or Journal of Bone and Joint Surgery . The publication rate of "best papers" at the POSNA annual meeting was found to be higher than rates reported for abstracts presented at the annual meetings of POSNA, American Society for Surgery of the Hand and European Pediatric Orthopaedic Society, but similar to the rates observed for American Association of Hip and Knee Surgeons, American Academy of Orthopaedic Surgeons, and Orthopaedic Trauma Association. Most of the selected "best papers" at the POSNA annual meeting are published and have a substantial impact on pediatric orthopaedic surgery practice.

LEVEL OF EVIDENCE

Level IV.

No difference in abstract publication rates between the open and closed American Shoulder and Elbow Surgeons meetings from 2013 to 2019

BACKGROUND

The American Shoulder and Elbow Surgeons (ASES) society has advanced the practice of shoulder and elbow care through the exhibition of research at academic meetings. The ASES annual meeting is a closed (member-only) conference annually held in October, while the specialty day is an open (non-members included) event that takes place during the American Academy of Orthopaedic Surgeons (AAOS) meeting week in March. This study aims to compare the rate of publication for abstracts presented at the open and closed ASES meetings from 2013 to 2019.

METHODS

The ASES website was searched to obtain the annual meeting and specialty day program agendas from 2013 to 2019. A standardized search protocol was employed to identify conference abstracts that went on to be published. Publications associated with an ASES abstract were analyzed through several variables including the time to publication, journal impact factor (JIF), and level of evidence.

RESULTS

There was no difference between the rates of publication of the open (76.5%, 121/158) and closed (75.3%, 223/296) meetings (P = .904). The median time to publication significantly differed between the open (7 months, 95% confidence interval [CI]: 5.0-10.0) and closed (11 months, 95% CI: 9.0-13.0) meetings (P = .02). There was no difference between the median JIF between the open (2.69, 95% CI: 2.41-2.81) and closed (2.73, 95% CI: 2.41-2.81) meetings. The distribution of the level of evidence in published articles comparing the open and closed meetings did not differ significantly (P = .446).

DISCUSSION

The overall quality of academic research presented at orthopedic subspecialty conferences can be objectively evaluated through abstract publication rates. Our analysis demonstrates that there is not a single significant difference among the publication rates, median JIF, and level of evidence distribution between the ASES open and closed meetings from 2013 to 2019. Impactful research is showcased at both the open and closed meetings. Societies that limit submissions from members only at annual meetings can consider soliciting research from non-members. While the quality of research would not decline if non-ASES members were invited to participate, the presence of a closed annual meeting may be a valuable tool for societies to expand their reach through member-exclusive benefits.

Quantifying the Level of Evidence of Podium Presentations at the American Association of Hip and Knee Surgeons From 2015 to 2019

BACKGROUND

The American Association of Hip and Knee Surgeons (AAHKS) Annual Scientific Meeting is a leading forum for the presentation and dissemination of research regarding the management of hip and knee pathology making research presented at these meetings a representation of the current literature in the field. The purpose of this study was to quantify the level of evidence of podium presentations presented at the AAHKS annual meeting from 2015 to 2019.

METHODS

Two reviewers evaluated the abstracts for the available presentations. Basic science and biomechanical studies were excluded from the review. Economic studies that were not able to be evaluated based on the American Academy of Orthopedic Surgeon guidelines were also excluded. The two reviewers then independently evaluated each abstract and assigned a level of evidence (level I-V) based on the American Academy of Orthopedic Surgeon classification scheme.

RESULTS

A total of 258 podium presentations were included. In total, 17 (7%) abstracts were graded level I evidence, 57 (22%) were graded level II, 85 (33%) were graded level III, and 98 (38%) were graded level IV (Table 1). There was a significant change in the distribution of the level of evidence of podium presentations over time (χ² = 24.6, P = .02). The proportion of level I studies has increased between 2015 and 2019 (from 3.9% to 11.8%) with a concomitant decrease in level IV studies (from 42.3% to 21.6%) over that time period.

CONCLUSIONS

There has been a significant improvement in the levels of evidence of podium presentations at the AAHKS Annual Meeting from 2015 to 2019.

Assessing the Quality of Evidence Presented at the Annual Conferences of Diabetes Canada

OBJECTIVES

Annual general meetings (AGMs) are often seen and promoted as great sources of contemporary information for the modern health-care professional. The quality of research evidence presented at these conferences, however, remains unclear. This paper evaluates the level of evidence (LoE) of research presented at the 2015 to 2019 AGMs of Diabetes Canada (DC).

METHODS

Using the framework we first published to evaluate the LoE presented at the Canadian Society of Nephrology (CSN) AGMs, 2 authors independently assigned eligible abstracts a study type and grade. Research assistants separately recorded corresponding author and city affiliations.

RESULTS

Of 832 published abstracts, 68% (N=568) met the inclusion criteria. These abstracts were classified as follows: 12% Level I (highest quality); 30% Level II; 36% Level III; 14% Level IV and 8% Level V (lowest quality). The LoE remained consistent over the 5-year study period, as observed by the Jonckheere-Terpstra test (p=0.754). The LoE of DC AGMs was similar to that of CSN AGMs (p=0.125). The number of authors was positively associated with improved LoE (p<0.001).

CONCLUSIONS

Based on our novel auditing methodology, the LoE of DC AGMs is on par with other analyzed national AGMs. Adoption of this metric for all AGMs would aid in identifying unfavourable trends and alert planning and abstract selection committees of trends in LoE. Such studies will increase transparency for stakeholders and facilitate quality improvement initiatives.

Clinical Level of Evidence Presented at the Cervical Spine Research Society (CSRS) Annual Meeting Over 10 Years (2008-2017): A Systematic Review

STUDY DESIGN

Systematic review.

OBJECTIVE

We systematically reviewed the level of clinical evidence presented at Cervical Spine Research Society annual meetings from 2008 through 2017.

SUMMARY OF BACKGROUND DATA

The Cervical Spine Research Society is dedicated to advancing knowledge of the cervical spine to promote evidence-based care. Research presented at these meetings impacts clinical practice.

METHODS

A total of 774 paper abstracts presented at Cervical Spine Research Society (CSRS) annual meetings were independently assessed by two reviewers. Reviewers designated a clinical level of evidence (LOE) to each included abstract from level I to level IV based on criteria set forth by the Oxford Centre for Evidence-Based Medicine. Reviewer agreement was assessed using Cohens Kappa coefficient (k) and disagreements were discussed until a consensus was reached. Wilcoxon rank sum test was used to assess for differences in LOE grades. Chi-squared testing was used to assess nonrandom changes in level of evidence and in excluded studies.

RESULTS

A total of 583 abstracts were included. Over the last 10 CSRS meetings, 5.15% of presentations were level I, 27.8% level II, 27.4% level III, and 39.6% level IV. The average LOE from 2008 to 2017 was 3.02 (median = 3). Additionally, 49.7% were therapeutic studies, 37.6% prognostic studies, and 12.7% diagnostic studies. When comparing the first 5 years (2008-2012) to the last 5 years (2013-2017), we observed a significant increase in Level II (P = 0.007) evidence and a corresponding decrease in level IV evidence (P < 0.001). The average LOE improved from 3.14 (2008-2012) to 2.91 (2013-2017); there was a significant improvement in LOE between the two periods (P = 0.001).

CONCLUSION

Emphasis on evidence-based medicine within cervical spine research has positively influenced the clinical level of evidence disseminated at CSRS annual meetings between 2008 and 2017. Continued focus on higher quality Level I studies is warranted.

LEVEL OF EVIDENCE

4.

Ten Years of Sports Health: Authorship Characteristics and Levels of Evidence

CONTEXT

Sports Health: A Multidisciplinary Approach, now 10 years into production, has been ranked a top-25 journal in sport sciences and has tripled its impact throughout its existence.

OBJECTIVE

To evaluate authorship trends and levels of evidence (LOE) of articles published in Sports Health from 2009 to 2018. The secondary aim was to analyze funding sources and internationalization throughout the journal's tenure.

DATA SOURCES

All clinical studies published in Sports Health between the years 2009 and 2018 were examined.

STUDY SELECTION

All publications from the provided years were electronically reviewed by 2 reviewers and evaluated for inclusion criteria. Editorials, society news, memorials, letters to the editor, and corrigenda were excluded.

STUDY DESIGN

Systematic review.

LEVEL OF EVIDENCE

Level 5.

DATA EXTRACTION

Articles were examined for number of authors, presence of female authorship, funding, country of origin, international collaboration, academic degree or certification of first and senior authors, and LOE. Clinical articles were assigned LOE based on guidelines from the University of Oxford's Centre for Evidence-Based Medicine.

RESULTS

A total of 654 articles were examined. The percentage of high-LOE studies increased throughout the study period. The percentage of publications with female authors also increased throughout the study period. The mean number of authors per article increased from 3.2 to 4.6 over the 10-year period (P < 0.05). The percentage of publications with international collaboration stayed consistent, while the number of countries per year increased during the study period. Overall, institutions from 23 countries have published in Sports Health since its inception to the time of this study.

CONCLUSION

Female authorship in Sports Health surpasses industry standards, and the percentage of high-LOE studies remains remarkably high. Sports Health has stayed true to its multidisciplinary scope, as evidenced by the authors' varying degrees and numerous countries that publish in the journal.

Level of Evidence and Publication Rates of Abstracts Presented at the American Society for Surgery of the Hand Annual Meetings: Comparison Over 23 Years

PURPOSE

The primary aims of this study were to determine how level of evidence and publication rates of American Society for Surgery of the Hand (ASSH) abstracts presented at the national meeting have changed over the past 23 years.

METHODS

Abstracts presented at the ASSH annual meeting from 1992 to 2014 were reviewed. Level of evidence (LoE) and publication status for each abstract were recorded. We calculated annual and overall LoE, publication rates, average time to publication, and top journals of publication for abstracts presented from 1992 to 2014. The LoE was categorized into level 1 or 2 studies, levels 3 to 5 studies, or nonclinical study.

RESULTS

A total of 1,757 abstracts were presented at ASSH meetings from 1992 to 2014; 942 abstracts were published in peer-reviewed journals for an overall publication rate of 53.6%. There was a significant increase in the proportion of levels 1 to 2 LoE abstracts over time (18% in 2007-2014 vs 11% in 1999-2006 and 2% in 1992-1998). There was also a significantly higher percentage of abstracts published over time (62% in 2007-2014 vs 52% in 1999-2006 and 47% in 1992-1998). Levels 1 to 2 LoE studies were associated with higher publication rates than nonclinical or levels 3 to 5 LoE studies.

CONCLUSIONS

This research provides historical trends on the LoE of abstracts presented at the ASSH annual meetings. Our study shows there are increasing numbers of levels 1 to 2 studies as well as higher publication rates of abstracts presented at more recent ASSH annual meetings. Levels 1 to 2 studies are more likely to be published than nonclinical or levels 3 to 5 studies.

CLINICAL RELEVANCE

Although not all questions can be feasibly answered with level 1 or level 2 studies, authors should continue to search for ways to strengthen study designs, producing more valid and comparable results with increased likelihood of publication driving forward the quality of hand surgery research. Higher recent publication rates may be partially due to the increased number of available journals for publication.

Clinical Level of Evidence Presented at the Lumbar Spine Research Society (LSRS) Annual Meeting Over 10 Years (2008-2017): A Systematic Review

STUDY DESIGN

A systematic review.

OBJECTIVE

The Lumbar Spine Research Society (LSRS) is dedicated to advancing knowledge of the lumbar spine to promote evidence-based care. We sought to systematically review the level of clinical evidence presented at LSRS annual meetings from 2008 through 2017.

SUMMARY OF BACKGROUND DATA

Improvements in clinical evidence have been reported at similar bone and joint scientific meetings.

METHODS

A total of 458 paper abstracts presented at LSRS annual meetings were independently assessed by two reviewers. Only clinical studies being included for analysis. Reviewers designated a clinical level of evidence (LOE) to each included abstract from level I to level IV based on criteria set forth by the Oxford Centre for Evidence-Based Medicine. Reviewer agreement was assessed using Cohens Kappa coefficient (k). Student t test was used to assess for differences in mean LOE grades. Chi-squared testing was used to assess nonrandom changes in LOE.

RESULTS

A total of 299 abstracts met inclusion criteria. Over the last 10 LSRS meetings, 2.68% of the presentations were level I, 22.4% were level II, 37.1% were level III, and 37.8% were level IV. We found the average LOE from 2008 to 2017 to be 3.10 (median = 3). In addition, 63.9% presentations were Therapeutic Studies, 30.1% were Prognostic Studies, and 6.02% were Diagnostic studies. When comparing the first 5 years (2008-2012) to the last 5 years (2013-2017), we observed a significant increase in Level II (P < 0.05) and Level III (P < 0.05) evidence along with a corresponding decrease in level IV evidence (P < 0.01). The average LOE improved significantly from 3.28 (2008-2012) to 2.88 (2013-2017) (P < 0.001).

CONCLUSION

Emphasis on evidence-based medicine within spine surgery, specifically pertaining to the lumbar spine, has positively influenced the clinical LOE disseminated at LSRS annual meetings between 2008 and 2017.

LEVEL OF EVIDENCE

4.

Assessing the Quality of Evidence Presented at Annual General Meetings: A 5-Year Retrospective Study

INTRODUCTION

Health care professionals rely on annual general meetings (AGMs) to obtain up-to-date information and practice guidelines relevant to their specialty. The majority of such information at meetings is presented through abstract sessions. However, the quality of the evidence presented during such abstract sessions is unclear. Standardized measures were applied to assess the quality of evidence of abstracts presented at the Canadian Society of Nephrology AGM over a 5-year period.

METHODS

Two authors independently reviewed all CSN AGM abstracts presented from 2012 to 2016. Using a schema published in 2011 by the Oxford Centre for Evidence-Based Medicine (OCEBM), each abstract was subsequently ranked based on the quality of evidence. Schema categories ranged from level I, representing the highest evidence quality, to level V, representing the lowest. The number of authors and the authors' institution affiliations were also collected from the abstracts, where available, or if affiliations were unclear, an internet search of the author was performed.

RESULTS

Six hundred forty-two articles were screened. In total, 70% (n = 450) met the inclusion criteria. When assessed, 15% of articles were level I (highest quality), 17% level II, 53% level III, 12% level IV, and 3% level V (lowest quality). A Jonckheere-Terpstra test demonstrated a significant trend of increasing quality of evidence (P < .05) and collaboration (P < .005) over the 5-year study period. There was a significant correlation between level of evidence and collaboration across years reviewed in the study, rs(98) = -0.226, P < .001.

DISCUSSION

The results indicate a consistent increase in quality of evidence and collaborative submissions over time. To the authors' knowledge, this is the first assessment and analysis of AGM presentation quality within internal medicine and its subspecialties. Documenting and monitoring changes in the quality of evidence with a standardized framework may offer valuable insight pertaining to the medical field and the research community.

An In-Depth Analysis of Publication Characteristics of Podium Presentations at the Arthroscopy Association of North America Annual Meetings, 2011-2014

PURPOSE

To determine the publication rate of abstracts presented at the annual meetings of the Arthroscopy Association of North America (AANA) from 2011 through 2014 and to compare the level of evidence (LoE) between published and unpublished studies.

METHODS

A comprehensive search was performed using PubMed and Google Scholar for all abstracts given as podium presentations at the 2011 through 2014 AANA Annual Meetings. A publication rate was calculated from all presented abstracts. Each abstract was reviewed to determine the study's LoE (I-V or nonclinical). Among published studies, the average time from presentation to publication was calculated. The journals in which these studies were published were also noted.

RESULTS

A total of 290 abstracts were given as podium presentations at AANA Annual Meetings from 2011 through 2014. Of the 290 studies presented, 195 (195/290, 67%) were published in peer-reviewed journals. Of the 195 published studies, 184 (184/195, 94%) were published within 3 years of the meeting date. Studies were most frequently published in Arthroscopy (n = 59) and the American Journal of Sports Medicine (n = 48). The average time from presentation to publication was 12.2 months. Overall, there was no significant difference between published and unpublished studies in terms of LoE (P = .24).

CONCLUSIONS

In recent years, studies presented at AANA Annual Meetings have achieved a high publication rate (67%). Based on other studies, this publication rate is comparable to recent American Orthopaedic Society for Sports Medicine (AOSSM) Annual Meetings and is higher than that of AANA Annual Meetings from previous years. The level of evidence of presented studies does not necessarily correlate with eventual publication.

CLINICAL RELEVANCE

The publication rate and level of evidence of podium presentations at AANA demonstrate the scientific impact the annual meeting has for peers pursuing orthopaedic research.

Level of evidence and authorship trends of clinical studies in knee surgery, sports traumatology, arthroscopy, 1995-2015

PURPOSE

There is increasing emphasis on publication quality and internationalization of author groups in orthopaedic literature. The purpose of this review was to evaluate the type of studies and the level of evidence (LOE) published in knee surgery, sports traumatology, arthroscopy (KSSTA) from 1995 to 2015. The secondary aim was to analyze trends in authorship characteristics in KSSTA.

METHODS

Two reviewers reviewed the table of contents of KSSTA and identified original papers from 1995, 2000, 2005, 2010, and 2015. The reviewers graded LOE from Levels I to IV using guidelines from the University of Oxford's Centre for Evidence-Based Medicine. For each article, the total number of authors and country of author group were also analyzed.

RESULTS

A total of 880 papers were analyzed. The proportions in LOE have stayed consistent throughout the study period (n.s.). There has been a significant increase in the number of published articles and the number of Level I and II studies (P < 0.01). Therapeutic articles were the most common type. The mean number of authors per KSSTA article significantly increased from 3.9 to 5.7 over the 20-year period (P < 0.01). The number of represented countries increased yearly and academic institutions from 40 different nationalities published articles in the Journal. Of the examined years, the percent of articles with international collaboration was 17.6%.

CONCLUSION

The proportion of LOE I and II articles published in KSSTA remains consistently high. Therapeutic studies are the most frequently published articles. There is an increase in international groups publishing in KSSTA.

LEVEL OF EVIDENCE

IV.

Level of evidence of free papers presented at the European Society of Sports Traumatology, Knee Surgery and Arthroscopy congress from 2008 to 2016

PURPOSE

The European Society of Sports Traumatology, Knee Surgery and Arthroscopy (ESSKA) congress is an important venue, and the research presented can be a critical source of information used to impact clinical decisions and health policies. The purpose of this study was to evaluate the level of evidence of clinical free papers presented at the ESSKA congress from 2008 to 2016. Moreover, this study evaluated whether there were any changes in the distribution of level of evidence over time.

METHODS

Two reviewers screened the free papers presented at the ESSKA biannual congresses 2008-2016 for clinical evidence. Clinical papers included observational studies and trials involving direct interaction between an investigator and human subjects. Biomechanical studies, technique demonstrations, cadaveric studies, and panel discussions were excluded. The reviewers independently graded their level of evidence from level I (e.g. high-quality randomized trials) to level IV (e.g. case series and reports) using the classification system published by the American Academy of Orthopaedic Surgeons.

RESULTS

Of 1036 free papers that were identified, 729 met the inclusion criteria and were evaluated. Overall, 18% of studies were level I, 24% level II, 25% level III, and 33% level IV evidence. There was a significant improvement in level of evidence over time (p < 0.0001), with the proportion of level I studies increasing most dramatically (9% in 2008, 20% in 2012, 24% in 2016). Free papers studying the knee had higher levels of evidence than those evaluating other joints (p = 0.002).

CONCLUSION

The level of evidence of clinical free papers presented at the ESSKA congress between 2008 and 2016 is high relative to other orthopaedic meetings. Moreover, there has been a significant improvement in the level of evidence over time.

LEVEL OF EVIDENCE

Systematic review, Level IV.

Publication Rates of Podium Presentation Abstracts at the Arthroscopy Association of North America Annual Meetings 2004-2012

PURPOSE

To determine the publication rates of podium presentation abstracts at the Arthroscopy Association of North America (AANA) annual scientific meetings from 2004 to 2012.

METHODS

A database of podium presentation abstracts at the annual meetings of the AANA was compiled. Abstract presentations that reached manuscript publication were determined by a PubMed search of the MEDLINE database and Google Scholar. The journal and publication date were then recorded for all identified published abstracts.

RESULTS

A total of 658 abstracts were selected for podium presentations at AANA annual meetings from 2004-2012 (range, 53-102 per year). Of these 658 abstracts, 443 (67.3%) went on to eventual publication in peer-reviewed journals. The mean time from the meeting to publication was 20.0 months. Most abstracts were published within 3 years of the meeting (n = 380, 85.8%), with a significant number of published abstracts reaching publication before the time of the meeting (n = 41, 9.3%). Published abstracts were most frequently published in Arthroscopy (n = 186, 42.0%), The American Journal of Sports Medicine (20.3%), and The Journal of Bone and Joint Surgery (6.1%).

CONCLUSIONS

The overall publication rate of podium presentations at AANA annual meetings (67.3%) was similar to publication rates for other major orthopaedic annual meetings. Most published abstracts (85.8%) were published within 3 years, and the mean time to publication was 20.0 months.

CLINICAL RELEVANCE

The rates of publication of podium presentations at AANA annual meetings show the impact and importance of these meetings in the advancement of orthopaedic research.

The Arthroscopy Journal Experience: Tailored for You and by You

As editors, we not only oversee the peer-review process to select the most timely and relevant content but also strive to group the articles we accept for publication in a way that best fulfills your expectations and needs as readers. The Internet has impacted not only how we review but also how we publish and how you read. We invite you to take advantage of Arthroscopy's many web-based features to tailor your reading experience.