An analysis of references used for the Orthopaedic In-Training Examination: what are their levels of evidence and journal impact factors?

Fifty Most-Cited Research Articles in Elbow Surgery: A Modern Reading List

Purpose

Bibliometric analysis is a common method for evaluating current trends within a scientific field. The primary aim of this study was to define and analyze the 50 most frequently cited articles in the field of elbow surgery, both of all time and those published during the 21st century.

Methods

We searched the Journal Citation Report to identify articles related to elbow surgery within academic journals. Articles were sorted by total citations. The overall top 50 articles and those published since 2000 were identified, and data were collected, including title, journal of publication, publication year, country of publication, citation density, level of evidence, article type, institution, and sex of the lead and senior authors, and inclusion on the reference list for the Orthopaedic In-Training Examination within the last 5 years. Descriptive statistics were reported, and correlation analysis was performed using Spearman test.

Results

For the most-cited elbow surgery articles, "fracture" was overall the most reported topic, whereas "lateral epicondylosis" and "fracture" were equal for those published since 2000. The United States was the most represented overall and for articles published since 2000. Women comprised 1/50 (2%) of lead authors overall, increasing to 8/50 (16%) for articles published during the 21st century. Most articles in during both periods contained level IV evidence, with level I evidence appearing infrequently (4%). Six percent of the most-cited articles of all time had appeared on the reference list of the Orthopaedic In-Training Examination within the past 5 years.

Conclusions

The top 50 most-cited elbow surgery articles often assess fracture and lateral epicondylosis, most commonly originating from the United States. Level IV retrospective series comprises over half of the articles on this list. Women remain underrepresented as authors.

Clinical Relevance

This study provides a modern reading list for upper-extremity surgeons about impactful elbow surgery articles.

The fifty most cited publications in shoulder arthroplasty research

INTRODUCTION

Frequency of citation provides one quantitative metric of the impact that an article has on a given field. The purpose of this study was to evaluate characteristics of the 50 most cited publications on shoulder arthroplasty.

METHODS

The Web of Science database was used to determine the 50 most frequently cited shoulder arthroplasty articles. Articles were evaluated for several factors including type of arthroplasty, citation frequency and rate, source journal, country of origin, study type, and level of evidence.

RESULTS

The most common countries of origin were the United States (60%) followed by France (24%) and Switzerland (6%). A total of 27 (54%) articles included anatomic total shoulder arthroplasty (TSA), 18 (36%) included reverse total shoulder arthroplasty (RTSA), and 15 (30%) included hemiarthroplasty. Articles including RTSA had the greatest number of citations compared to those on TSA (p = 0.037) and hemiarthroplasty (p = 0.035). Citations per year were also greatest with RTSA compared to TSA and hemiarthroplasty (p ≤ 0.001).

DISCUSSION

This citation analysis includes many of the landmark studies that shaped, and continue to impact, the field of shoulder arthroplasty. This list provides a group of influential articles that provide a foundation for future research in shoulder arthroplasty.

Cross-Sectional Analysis of Foot and Ankle Questions on the Orthopaedic In-Training Examination: A Guide for Resident Preparation

Background:

The Orthopaedic In-Training Examination (OITE) is a standardized examination administered annually to orthopaedic surgery residents. The examination is designed to evaluate resident knowledge and academic performance of residency programs.

Methods:

All OITE foot and ankle questions from 2009 through 2012 and 2017 through 2020 were analyzed. Subtopics, taxonomy, references, and use of imaging modalities were recorded.

Results:

There were a total of 167 foot and ankle (F&A)–related questions across 8 years of OITE examinations. Trauma remained the most commonly tested subtopic of F&A across both subsets, followed by rehabilitation, tendon disorders, and arthritis. We found an increase in questions related to arthritis (P = .05) and a decrease of questions related to the diabetic foot (P = .02). Taxonomy 3 questions constituted 49.5% of F&A questions from 2009 through 2012 compared with 44.7% of questions from 2017 to 2020 (P = .54). Radiography was the most commonly used imaging modality in both subsets. From 2009 to 2012, 63.6% of questions included a radiograph compared with 76.5% in 2017 through 2020 (P = .13). FAI (Foot & Ankle International), JAAOS (Journal of the American Academy of Orthopaedic Surgeons), and JBJS (The Journal of Bone and Joint Surgery) were the most commonly cited journals, making up more than 50% of total citations. Citations per question increased from 2.20 to 2.42 from 2009-2012 to 2017-2020 (P = .01). The average lag time in the early subset was 8.2 years and 8.9 years in the later subset.

Conclusion:

This study provides a detailed analysis of the F&A section of the OITE. Use of this analysis can provide residents with a guide on how to better prepare for the OITE examination.

Level of Evidence:

Level IV, cross-sectional review of Orthopaedic In-Training Examination questions

What are the health consequences associated with differences in medical malpractice liability laws? An instrumental variable analysis of surgery effects on health outcomes for proximal humeral facture across states with different liability rules

BACKGROUND

States enacted tort reforms to lower medical malpractice liability, which are associated with higher surgery rates among Medicare patients with shoulder conditions. Surgery in this group often entails tradeoffs between improved health and increased risk of morbidity and mortality. We assessed whether differences in surgery rates across states with different liability rules are associated with surgical outcomes among Medicare patients with proximal humeral fracture.

METHODS

We obtained data for 67,966 Medicare beneficiaries with a diagnosis of proximal humeral fracture in 2011. Outcome measures included adverse events, mortality, and treatment success rates, defined as surviving the treatment period with < $300 in shoulder-related expenditures. We used existing state-level tort reform rules as instruments for surgical treatment and separately as predictors to answer our research question, both for the full cohort and for stratified subgroups based on age and general health status measured by Charlson Comorbidity Index and Function-Related Indicators.

RESULTS

We found a 0.32 percentage-point increase (p < 0.05) in treatment success and a 0.21 percentage-point increase (p < 0.01) in mortality for every 1 percentage-point increase in surgery rates among patients in states with lower liability risk. In subgroup analyses, mortality increased among more vulnerable patients, by 0.29 percentage-point (p < 0.01) for patients with Charlson Comorbidity Index >  = 2 and by 0.45 percentage-point (p < 0.01) among those patients with Function-Related Indicator scores >  = 2. On the other hand, treatment success increased in patients with lower Function-Related Index scores (< 2) by 0.54 percentage-point (p < 0.001). However, younger Medicare patients (< 80 years) experienced an increase in both mortality (0.28 percentage-point, p < 0.01) and treatment success (0.89 percentage-point, p < 0.01). The reduced-form estimates are consistent with our instrumental variable results.

CONCLUSIONS

A tradeoff exists between increased mortality risk and increased treatment success across states with different malpractice risk levels. These results varied across patient subgroups, with more vulnerable patients generally bearing the brunt of the increased mortality and less vulnerable patients enjoying increased success rates. These findings highlight the important risk-reward scenario associated with different liability environments, especially among patients with different health status.

Orthopaedic In-training Exam Preparation among Orthopaedic Surgery Residency Programs

OBJECTIVE

The Orthopaedic In-Training Examination (OITE) is given annually to residents to assess their knowledge of clinical orthopaedics and orthopaedic basic science. This study sought to determine what resources orthopaedic residency programs recommend and/or require for residents as preparative tools for the OITE and to understand which resources are most beneficial.

DESIGN

An anonymous electronic survey was distributed to program directors of orthopaedic surgery residency programs. It included questions about resources that program directors recommended or required residents to use when preparing for the OITE.

SETTING

The survey was prepared and the results analyzed at two academic medical institutions in Louisiana. The survey was available to respondents in December 2019 and January 2020.

PARTICIPANTS

The survey was delivered to program directors of 148 ACGME accredited orthopaedic surgery residency programs in the United States.

RESULTS

148 orthopaedic surgery residency program directors received the survey and 44 responded (response rate: 29.7%). Twenty-nine programs (73.2%) reported that practice tests/review of old OITE exams positively correlated with better scores. The most recommended resources for OITE preparation were Orthobullets (35 programs; 85%) and Res Study (AAOS) (27 programs; 67.5%). Programs having formal/required preparation programs had lower mean overall scores on the OITE than programs that did not. Those without a formal/required program scored in the 60th percentile, while those with a formal/required program scored in the 53.3^rd percentile, a difference of 6.7 percentile points (p = 0.049). The mean overall percentiles were significantly higher for programs that reported having a threshold/goal OITE score for residents compared to those that didn't. Programs with a threshold/goal score scored in the 60.6^th percentile on average, while those without a threshold/goal score scored in the 51.7^th percentile on average, a difference of 8.9 percentile points (p = 0.0095).

CONCLUSIONS

There is substantial variability in the resources that are utilized in preparation for the OITE, with the most commonly recommended resource being Orthobullets. The study method most cited as having a positive impact on scores was practice tests/review of old OITE exams. Having a formal/required program resulted in programs having lower OITE scores. Setting a goal/threshold score correlates positively with an increase in OITE score. Encouraging residents to use practice questions/old OITE tests, setting a goal/threshold score and avoiding formal/required preparation programs may improve resident performance on the OITE.

State medical malpractice laws and utilization of surgical treatment for rotator cuff tear and proximal humerus fracture: an observational cohort study

Background

How much does the medical malpractice system affect treatment decisions in orthopaedics? To further this inquiry, we sought to assess whether malpractice liability is associated with differences in surgery rates among elderly orthopaedic patients.

Methods

Medicare data were obtained for patients with a rotator cuff tear or proximal humerus fracture in 2011. Multivariate regressions were used to assess whether the probability of surgery is associated with various state-level rules that increase or decrease malpractice liability risks.

Results

Study results indicate that lower liability is associated with higher surgery rates. States with joint and several liability, caps on punitive damages, and punitive evidence rule had surgery rates that were respectively 5%-, 1%-, and 1%-point higher for rotator cuff tears, and 2%-, 2%- and 1%-point higher for proximal humerus fractures. Conversely, greater liability is associated with lower surgery rates, respectively 6%- and 9%-points lower for rotator cuff patients in states with comparative negligence and pure comparative negligence.

Conclusions

Medical malpractice liability is associated with orthopaedic treatment choices. Future research should investigate whether treatment differences result in health outcome changes to assess the costs and benefits of the medical liability system.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-021-06544-8.

Orthopaedic In-Training Examination: History, Perspective, and Tips for Residents

Introduced in 1963, the orthopaedic in-training examination (OITE) is a standardized, national test administered annually to orthopaedic residents by the American Academy of Orthopaedic Surgeons. The examination consists of 275 multiple-choice questions that cover 11 domains of orthopaedic knowledge, including basic science, foot and ankle, hand, hip and knee, oncology, pediatrics, shoulder and elbow, spine, sports medicine, trauma, and practice management. The OITE has been validated and is considered predictive of success in both orthopaedic surgery residency and on the American Board of Orthopaedic Surgery part I examination. This article provides a historical overview of the OITE, details its current structure and scoring system, and reviews currently available study materials. For examination preparation, the residents are encouraged to (1) start the examination preparation early, (2) practice on old OITE or self-assessment examination questions, (3) focus on the questions answered incorrectly, (4) focus on comprehension over memorization, and (5) recognize and avoid burnout. Finally, the residents should have a systemic way of approaching each multiple-choice question, both during practice and on the actual examination.

Updated Analysis of the Oncology Section of the OITE from 2007 to 2019

The Orthopaedic In-Training Exam (OITE) is administered annually to orthopedic surgery residents to assess their medical knowledge. The authors provide a comprehensive review of the orthopedic oncology portion of the exam in order to aid residents in preparation for future in-training and licensing examinations as well as to help guide oncology residency education curriculum. All of the orthopedic oncology questions on the OITE from 2007 to 2019 were reviewed. Analysis included (1) the number of oncology questions each year, (2) question topic, (3) question taxonomy (knowledge versus interpretation), (4) the type of imaging modalities (radiological, histological), (5) most commonly cited references, and (6) level of evidence. Descriptive statistics were utilized to compare means between variables. From 2007 to 2019, there was a total of 292 tumor-related questions with a mean of 22.5 tumor-related questions (range 19-28) per year. Of the questions, 54.8% pertained to malignant tumors and 45.2% to benign tumors. Assessment of question taxonomy showed that 79.8% of questions required interpretation of imaging and analysis of the information provided versus 20.2% of questions being knowledge recall type. Of the questions, 76.7% required interpretation of radiological images, pathological images, or both. Orthopaedic Knowledge Update, Journal of the American Academy of Orthopaedic Surgeons, and Journal of Bone and Joint Surgery were the three most commonly cited question sources. Only 29 (9.84%) oncology questions over the past 13 years have been supported by level I or II sources of evidence. Better understanding of the OITE make-up, question distribution, and number and style of question, reference sources can improve an orthopedic residents' performance as well as better guide educational curriculum to prepare residents in their orthopedic oncology education.

Analysis of the Sports Medicine Section of the Orthopaedic In-Training Examination: Improvements in Levels of Evidence and Question Taxonomy Over a 12-Year Period

There has been a recent shift within the orthopedic literature to publish articles with higher levels of evidence. In this investigation, the trends in question taxonomy and the levels of evidence of the references for sports medicine questions on the Orthopaedic In-Training Examination (OITE) during a 12-year period were evaluated. Sports medicine questions were obtained from the OITEs administered between 2005 and 2007 and between 2014 and 2016. The taxonomy of each question was characterized, and levels of evidence for all references were assigned using American Academy of Orthopaedic Surgeons guidelines. Question taxonomy and article levels of evidence from 2005 to 2007 were compared with those from 2014 to 2016. Sports medicine questions comprised 8% of the OITEs in both examination groups. The questions from 2014 to 2016 had a higher mean taxonomic level (2.26 vs 1.52, P=.0001) and a greater proportion of studies with high levels of evidence (levels 1 and 2) (21% vs 10%, P=.027). However, references with low levels of evidence or nonprimary resources made up 82.2% and 68.5% of the total references on the older and more recent examinations, respectively. References from 2014 to 2016 were, on average, 2 years older than those from earlier examinations. This study indicated that sports medicine questions on recent OITEs cite references of higher levels of evidence and contain higher taxonomic question structure than examinations 10 to 12 years ago. However, the majority of questions still cite articles with low levels of evidence or nonprimary sources. These findings can be used to guide resident education and continue improvements in the selection of references for questions on the Sports Medicine section of the OITE. [Orthopedics. 2020;43(5):e460-e464.].

Patient Attitudes Toward Resident and Fellow Participation in Orthopedic Surgery

BACKGROUND

Residents' and fellows' participation in orthopedic surgery is a potential source of anxiety and concern for patients. The purpose of this study was to determine patients' attitudes toward trainee involvement in orthopedic surgery, surgeons as educators, and disclosure of trainee involvement.

METHODS

Three hundred two consecutive patients with preoperative and postoperative appointments at three arthroplasty practices in academic medical centers were surveyed with an anonymous, self-administered questionnaire. The questionnaire was developed in consultation with an expert in survey design.

RESULTS

Two hundred thirty-four patients completed the questionnaire (response rate 77.5%). Respondents were 60.5% female, 79.6% white, 66.5% privately insured, and 82.8% had at least some college education. About 65.9% of the respondents felt that surgeons who teach are better surgeons. Nearly all felt residents and fellows should perform surgeries as part of their education (94.1% and 95.3%, respectively). However, 39.7% of the respondents were not satisfactory with a second-year resident assisting in their own surgery. Patients dissatisfied with their most recent orthopedic surgery were more likely to respond that they did not want residents helping with their surgery. Respondents agreed that resident or fellow involvement in surgery should be disclosed (92.2% and 90.1%, respectively).

CONCLUSIONS

Insured and educated patients in the United States overwhelmingly desire disclosure of trainee involvement in their surgery. To address the need for orthopedic training in the context of a patient population that is not fully comfortable with trainee involvement in their own surgery, an open discussion between patients and surgeons regarding trainees' roles may be the best course of action.

Orthopedic In-Training Examination: An Analysis of the Sports Medicine Section-An Update

OBJECTIVE

Previously published studies have evaluated Orthopedic In-Training Examination sports medicine questions, but none have evaluated whether question difficulty has changed over time.

DESIGN

Sports medicine subsection questions between 2012 and 2016 were evaluated and compared with previously published data on Orthopedic In-Training Examination from 2004 to 2009. Question categories were classified into 1 of 3 taxonomy levels-basic recall, diagnosis, and advanced problem solving.

SETTING

Medical University of South Carolina; Charleston, SC, 29425; Institution.

PARTICIPANTS

Two residents evaluated the Sports Subsection questions separately. Then an attending physician evaluated the questions to resolve discrepancies. A statistician was used for analysis.

RESULTS

Utilization of imaging modalities averaged 37%, with 28% (11/39) of the questions containing 2 or more imaging modalities. There were increases in utilization of advanced problem-solving questions (45% vs. 27%, p = 0.002) and decreases in basic recall questions (49% vs. 67%, p = 0.008) compared with previously published data.

CONCLUSIONS

While the percentage of the Orthopedic In-Training Examination represented by sports medicine has remained relatively unchanged, there were fewer questions requiring residents to demonstrate simple recall and diagnosis, and increased demand to perform advanced problem solving while utilizing multiple imaging modalities.

An Analysis of the Most Commonly Tested Topics and Their Taxonomy From Recent Self-Assessment Examinations

OBJECTIVE

The purpose of this study is to determine the most commonly tested topics and the question taxonomy of the American Academy of Orthopaedic Surgeons Self-Assessment Examinations (SAE) from 2009 through 2014.

DESIGN

All SAEs were analyzed from 2009 through 2014. The SAEs were separated by subject and the questions of each SAE were analyzed for topic, taxonomic classification, and question type.

RESULTS

A total of 2107 questions were reviewed from 10 different subjects. In all, 6 of the 9 subjects had roughly 1/3 of their questions composed of the 3 most commonly tested topics. Each subject had at least 1 trauma-related question within its top 5 most commonly tested topics. Almost half (47%) of all questions were of taxonomy 1 classification and 29% were taxonomy 3. The Basic Science SAEs had the greatest percentage of taxonomy 1 questions of any subject (83%) whereas Trauma contained the highest percentage of taxonomy 3 questions (47%).

CONCLUSIONS

Certain topics within each subject are consistently tested more often than other topics. In general, the 3 most commonly tested topics comprise about one-third of total questions and orthopedic surgeons should be very familiar with these topics in order to best prepare for standardized examinations.

Levels of Evidence for Hand Questions on the Orthopaedic In-Training Examination

Background: Although analyses of the Orthopaedic In-Training Examination (OITE) subspecialty content domains have been performed, few studies have analyzed the levels of evidence (LoEs) for journal articles used as references to create OITE questions. We present an analysis of reference characteristics and question taxonomy for the hand surgery content domain on the OITE. We aim to determine whether level of evidence (LoE) for hand surgery questions have increased over a 15-year period. Methods: All questions and references in the hand surgery content domain on the OITE from 1995-1997 and 2010-2012 were reviewed. The taxonomic classification was determined for each question. Publication characteristics were defined for each reference, and each primary journal article was assigned a LoE. Results: A total of 129 questions containing 222 references met inclusion criteria: 76 questions from 1995-1997 and 53 from 2010-2012. The Journal of Hand Surgery American and European Volumes, Journal of Bone and Joint Surgery American and British Volumes, and the Journal of the American Academy of Orthopaedic Surgeons were the most frequently cited journals overall. Recent examinations were more likely to have Buckwalter T3 complex clinical management questions. There was a statically significant increase in the LoE used to create hand questions on the 2010-2012 compared with the 1995-1997 OITE. Conclusions: Primary journal articles cited on the hand surgery content domain of the OITE frequently included recent publications from both general and subspecialty journals. More recent examination questions appear to test clinical management scenarios. LoE for hand questions has increased over a 15-year period. Our results can be used as a guide to help prepare orthopedic residents for the OITE.

The 50 Most Cited Articles in Rotator Cuff Repair Research

Analysis of the number of citations within a given specialty provides information on the classic publications of that specialty. The goals of this study were to identify the 50 most cited articles on rotator cuff repair and to analyze various characteristics of these articles. The ISI Web of Science (Thomson Reuters, Philadelphia, Pennsylvania) was used to conduct a search for the term rotator cuff repair. The 50 most cited articles were retrieved, and the following objective characteristics of each article were recorded: number of times cited, citation density, journal, country of origin, and language. The following subjective characteristics of each article were also recorded: article type (clinical vs basic science), article subtype, and level of evidence for clinical articles. Of the 50 most cited articles on rotator cuff repair, the number of citations ranged from 138 to 677 (mean, 232±133 citations) and citation density ranged from 3.8 to 53.5 citations per year (mean, 16.9±9.2 citations per year). The articles were published between 1974 and 2011, with most of the articles published in the 2000s (29 articles), followed by the 1990s (16 articles). The articles originated from 8 countries, with the United States accounting for 30 articles (60%). Overall, 66% of the articles were clinical and 34% were basic science. The most common article subtype was the clinical case series (48%). Of the 33 clinical articles, 24 (73%) were level IV. Among the 50 most cited articles on rotator cuff repair, the case series was the most common article subtype, showing the effect that publication of preliminary outcomes and new surgical techniques has had on surgeons performing rotator cuff repair. [Orthopedics. 2016; 39(6):e1045-e1051.].

Levels of Evidence for Foot and Ankle Questions on the Orthopaedic In-Training Examination: 15-Year Trends

BACKGROUND

The foot and ankle (FA) content domain is a component of the orthopaedic in-training examination (OITE). Levels of evidence (LoE) have been infrequently studied on the OITE. The purpose of this study is to determine if LoE for primary journal articles referenced for FA questions increased over a 15-year period. We also aim to determine if reference characteristics and question taxonomy have changed during this period.

METHODS

All 132 questions and 261 references in the FA content domain from 1995 to 1997 and from 2010 to 2012 were included. We defined the characteristics of each reference and taxonomy of each question. Every primary journal article was assigned a LoE based on American Academy of Orthopaedic Surgeons (AAOS) guidelines.

RESULTS

Foot & Ankle International (FAI) was the most frequently cited journal. The change in the distribution of the Buckwalter classifications was statistically significant (p = 0.0286) with an increase in the number of clinical management questions. There were more level I studies on the 2010 to 2012 OITE (p = 0.0478) 6/54 (11%) of questions on the 2010 to 2012 OITE cited level I or II evidence compared with 3/78 (4%) on the 1995 to 1997 examinations (p = 0.1035).

CONCLUSIONS

There is a trend toward improved LoE for journal articles within the FA content domain on the OITE over a 15-year period, particularly when analyzing the increase in level I studies. FAI is the most frequently cited journal and questions increasingly test clinical management concepts.

CLINICAL RELEVANCE

Our results can be used to help improve resident self-study and suggest that reviewing recent FAI articles may aid OITE preparation.

LEVEL OF EVIDENCE

Basic Science.

Relevancy of an In-Service Examination for Core Knowledge Training in a Surgical Subspecialty

OBJECTIVE

To facilitate knowledge acquisition during plastic surgery residency, we analyzed the breast curriculum on the Plastic Surgery In-Service Training Exam (PSITE).

DESIGN

Breast-related questions on 6 consecutive PSITEs were analyzed (2008-2013). Topics were categorized by the content outline for the American Board of Plastic Surgery written board examination. Question vignettes were classified by taxonomy and clinical setting. References for correct answer choices were categorized by source and publication lag.

RESULTS

A total of 136 breast-related questions were analyzed (136/1174, 12%). Questions tended to appear more in the Breast and Cosmetic (75%) section than the Comprehensive (25%) section (p < 0.001). Most question vignettes were written in a clinical setting (64%, p < 0.001). Question taxonomy was evenly distributed among recall (34%), interpretation (28%), and decision-making (37%, p > 0.05). Only 6% of questions required photographic evaluation. Breast-related topics focused on esthetic problems (35%), traumatic deformities (22%), and tumors (21%). Answer references comprised 293 citations to 63 unique journals published a median of 6 years before PSITE administration. Plastic and Reconstructive Surgery (57%) was the most cited journal (p < 0.001) and Surgery of the Breast: Principles and Art by Spear was the most referenced textbook (22%).

CONCLUSIONS

The PSITE affords a curriculum that reflects breast-related topics on the American Board of Plastic Surgery written board examination. These data may optimize knowledge acquisition in esthetic and reconstructive breast surgery.

An assessment of the Chilean National Examination of Orthopaedic Surgery

BACKGROUND

The Chilean National Examination of Orthopaedic Surgery (EMNOT) has been administered since 2009. It was developed to determine whether individual residents are meeting minimal knowledge standards at the end of their training programs.

METHODS

We performed a retrospective evaluation of the EMNOT for all years it has been administered (2009-2015). The test was analyzed for content, taxonomy of questions asked (1: direct recall; 2: diagnosis; 3: evaluation/decision-making), residents' performance, difficulty index and discrimination index.

RESULTS

During the years of EMNOT administration, the most frequently tested areas have been pediatric orthopaedics (22.9 %), spine (13.8 %), general orthopaedics (13.8 %) and musculoskeletal trauma (9.9 %). A significant increase in questions with images was observed, as well as a significant decrease in the percentage of Type 1 and an increase in Type 3 questions. The Difficulty Index showed a medium level of difficulty for all years the examination has been administered. The Discrimination Index showed good discrimination in 2009, fair discrimination from 2010 through 2012, and excellent discrimination from 2013 through 2015.

CONCLUSION

The EMNOT has evolved over several years to include better quality questions, better discrimination, and a more representative distribution of questions covering the different orthopaedic sub-specialties. This examination represents an effective instrument for quality assurance of orthopaedic residency programs in Chile.

Low Levels of Evidence on the Plastic Surgery In-Service Training Exam

BACKGROUND

The Plastic Surgery In-Service Training Exam is written by the American Society of Plastic Surgeons. Examinees reasonably infer that tested material reflects the Society's vision for the core curriculum in plastic surgery. The purpose of this study was to determine the levels of evidence on which credited answers to the examination questions are based.

METHODS

Two recent Plastic Surgery In-Service Training Exams (2014 and 2015) were analyzed. Questions were categorized using a taxonomy model. Recommended journal article references for Level III (decision-making) questions were assigned a level of evidence. Exam sections were analyzed for differences in question taxonomy distribution and level of evidence. To look for studies with higher levels of evidence, a PubMed search was conducted for a random sample of 10 questions from each section.

RESULTS

One hundred three Level I (25.8 percent), 138 Level II (34.5 percent), and 159 Level III (39.8 percent) questions were analyzed (p < 0.001). The hand and lower extremity section had the highest percentage of Level III questions (50.0 percent; p = 0.005). Journal articles had a mean level of evidence of 3.9 ± 0.7. The number of articles with a low level of evidence (IV and V) (p = 0.624) and the percentage of questions supported by articles with a high level of evidence (I and II) (p = 0.406) did not vary by section. The PubMed search revealed no instances of a higher level of evidence than the recommended reading list.

CONCLUSIONS

A significant percentage of Plastic Surgery In-Service Training Exam questions test clinical management, but most are supported with a low level of evidence. Although that is consistent with low level of evidence of plastic surgery literature, educators should recognize the potential for biases of question writers.

Analysis of References on the Plastic Surgery In-Service Training Exam

BACKGROUND

The Plastic Surgery In-Service Training Exam is a knowledge assessment tool widely used during plastic surgery training in the United States. This study analyzed literature supporting correct answer choices to determine highest yield sources, journal publication lag, and journal impact factors.

METHODS

Digital syllabi of 10 consecutive Plastic Surgery In-Service Training Exam administrations (2006 to 2015) were reviewed. The most-referenced articles, journals, and textbooks were determined. Mean journal impact factor and publication lag were calculated and differences were elucidated by section.

RESULTS

Two thousand questions and 5386 references were analyzed. From 2006 to 2015, the percentage of journal citations increased, whereas textbook references decreased (p < 0.001). Plastic and Reconstructive Surgery was cited with greatest frequency (38.5 percent), followed by Clinics in Plastic Surgery (5.6 percent), Journal of Hand Surgery (American volume) (5.1 percent), and Annals of Plastic Surgery (3.8 percent). There was a trend toward less publication lag over the study period (p = 0.05), with a mean publication lag of 9.1 ± 9.0 years for all journal articles. Mean journal impact factor was 2.3 ± 4.3 and lowest for the hand and lower extremity section (1.7 ± 2.8; p < 0.001). The highest yield textbooks were elucidated by section.

CONCLUSION

Plastic surgery faculty and residents may use these data to facilitate knowledge acquisition during residency.