Foot and ankle questions on the orthopaedic in-training examination: analysis of content, reference, and performance

Analysis of Questions in Sections of the Orthopaedic In-Training Examination: A Scoping Review

PURPOSE

The purpose of this review was to assess all available studies that analyzed the types of questions in individual sections of the Orthopaedic In-Training Examination, which may be used as a reference for residents studying for their examination.

METHODS

Following the Providing Innovative Service Models and Assessment extension for Scoping Reviews guidelines, a systematic review was conducted on studies that report on sections or question categories of the Orthopaedic In-Training Examination using PubMed, MEDLINE, and Web of Science databases. Two reviewers and an arbitrator reviewed and extracted relevant data from 20 included studies which made up the systematic review.

RESULTS

All 20 studies in the review reported the mean number of questions per section, with the highest coming from musculoskeletal trauma (18.9% to 19.0%). 18 studies reported the Buckwalter taxonomic classification; 42.0% of questions were T1, 18.2% were T2, and 39.5% were T3 with a wide range from section to section. Primary sources were nearly three times more likely to be cited when compared with textbook sources. There were 12 journals that were commonly cited with the most being the Journal of Bone and Joint Surgery: American Volume (17/18).

DISCUSSION

This study accurately portrays the characteristics of each section of the Orthopaedic In-Training Examination over the past 10 years. These data suggest that orthopaedic residents may be inclined to focus on musculoskeletal trauma, topics related to clinical management, and primary journal sources for studying. In addition, residency programs may choose to focus on higher yield sources or material to prepare their residents for the examination.

Cross-Sectional Analysis of Foot and Ankle Questions on the Orthopaedic In-Training Examination: A Guide for Resident Preparation

Background:

The Orthopaedic In-Training Examination (OITE) is a standardized examination administered annually to orthopaedic surgery residents. The examination is designed to evaluate resident knowledge and academic performance of residency programs.

Methods:

All OITE foot and ankle questions from 2009 through 2012 and 2017 through 2020 were analyzed. Subtopics, taxonomy, references, and use of imaging modalities were recorded.

Results:

There were a total of 167 foot and ankle (F&A)–related questions across 8 years of OITE examinations. Trauma remained the most commonly tested subtopic of F&A across both subsets, followed by rehabilitation, tendon disorders, and arthritis. We found an increase in questions related to arthritis (P = .05) and a decrease of questions related to the diabetic foot (P = .02). Taxonomy 3 questions constituted 49.5% of F&A questions from 2009 through 2012 compared with 44.7% of questions from 2017 to 2020 (P = .54). Radiography was the most commonly used imaging modality in both subsets. From 2009 to 2012, 63.6% of questions included a radiograph compared with 76.5% in 2017 through 2020 (P = .13). FAI (Foot & Ankle International), JAAOS (Journal of the American Academy of Orthopaedic Surgeons), and JBJS (The Journal of Bone and Joint Surgery) were the most commonly cited journals, making up more than 50% of total citations. Citations per question increased from 2.20 to 2.42 from 2009-2012 to 2017-2020 (P = .01). The average lag time in the early subset was 8.2 years and 8.9 years in the later subset.

Conclusion:

This study provides a detailed analysis of the F&A section of the OITE. Use of this analysis can provide residents with a guide on how to better prepare for the OITE examination.

Level of Evidence:

Level IV, cross-sectional review of Orthopaedic In-Training Examination questions

Analysis of the Basic Science Questions on the Orthopaedic In-Training Examination From 2014 to 2019

INTRODUCTION

The Orthopaedic In-Training Examination (OITE), produced by the American Academy of Orthopaedic Surgeons (AAOS), plays an important role in the educational mission of orthopaedic residency programs nationwide. An up-to-date understanding of this examination is critical for programs to develop an appropriate curriculum and for individuals to identify learning resources. This study presents an updated analysis of the basic science section of the OITE from 2014 to 2019.

METHODS

All questions from the OITE from 2014 to 2019 were reviewed. Each question from the basic science section was categorized by topic and taxonomy. The use of radiographic images or other clinical media was recorded. The reference section was analyzed for bibliometric factors. Pearson chi-square tests were used as appropriate for statistical comparison.

RESULTS

In total, 185 of 1,600 questions in the basic science section were used over the 6-year study period (11.6%). The proportion of basic science questions ranged from 10.7% to 12.0% from year to year. The most frequently tested topics were cellular and molecular biology (23.8%), physiology/pathophysiology (16.8%), and pharmacology (10.8%). There was an increase in the number of biostatistics questions from 2017 to 2019 compared with the number from 2014 to 2016 (P = 0.02). The most common taxonomic category was knowledge recall (89.7%). A total of 383 references were cited from 122 sources. The 3 most common sources accounting for 44.4% of all citations were produced by the AAOS.

DISCUSSION

The basic science section of the OITE accounts for approximately 11% of all questions, with the most common taxonomy being knowledge recall (89.7%). Recent tests have emphasized biostatistics, highlighting the importance of incorporating biostatistics into residency education. Reference materials produced by the AAOS were highly cited in this section.

Analysis of Hip and Knee Reconstruction Questions on the Orthopedic In-Training Examination

BACKGROUND

It is vital for orthopedic residents and residency programs to have a current understanding of the materials and resources utilized on the Orthopedic In-Training Examination (OITE) to tailor resident educational curricula accordingly. This study presents an updated analysis of the hip and knee section of the OITE.

METHODS

All OITE questions related to hip and knee reconstruction over six examinations between 2014 and 2019 were analyzed for topic, subtopic, taxonomy, imaging modalities, resident performance, and references.

RESULTS

There were 166 hip and knee reconstruction questions of 1600 OITE questions (10.4%) over a six-year period. The most commonly tested topics include mechanical properties of total knee and hip implants (10.8%), instability after THA (10.8%), periprosthetic fracture (10.2%), and prosthetic joint infection (10.2%). A total of 362 references were cited from 68 different sources. The most common sources were JOA, JBJS, JAAOS, and CORR, which were collectively responsible for 68% of all citations. There was an average publication lag of 7.1 years, with 75% of all citations falling within 10 years of the question date. Compared with a prior analysis from 2005 and 2009, there were significantly more complex multistep questions regarding treatment and fewer one-step knowledge recall questions (P = .003). Similarly, recent tests had significantly more questions involving interpretation of radiographs (55%, P < .001) and advanced imaging (9.6%, P < .001), compared with a decade ago.

CONCLUSIONS

The OITE continues to evolve over time, incorporating recent literature and topics. The current analysis identifies high-yield topics and resources that can guide resident preparation for the OITE hip and knee section.

Orthopedic In-Training Examination: An Analysis of the Sports Medicine Section-An Update

OBJECTIVE

Previously published studies have evaluated Orthopedic In-Training Examination sports medicine questions, but none have evaluated whether question difficulty has changed over time.

DESIGN

Sports medicine subsection questions between 2012 and 2016 were evaluated and compared with previously published data on Orthopedic In-Training Examination from 2004 to 2009. Question categories were classified into 1 of 3 taxonomy levels-basic recall, diagnosis, and advanced problem solving.

SETTING

Medical University of South Carolina; Charleston, SC, 29425; Institution.

PARTICIPANTS

Two residents evaluated the Sports Subsection questions separately. Then an attending physician evaluated the questions to resolve discrepancies. A statistician was used for analysis.

RESULTS

Utilization of imaging modalities averaged 37%, with 28% (11/39) of the questions containing 2 or more imaging modalities. There were increases in utilization of advanced problem-solving questions (45% vs. 27%, p = 0.002) and decreases in basic recall questions (49% vs. 67%, p = 0.008) compared with previously published data.

CONCLUSIONS

While the percentage of the Orthopedic In-Training Examination represented by sports medicine has remained relatively unchanged, there were fewer questions requiring residents to demonstrate simple recall and diagnosis, and increased demand to perform advanced problem solving while utilizing multiple imaging modalities.

An Analysis of the Most Commonly Tested Topics and Their Taxonomy From Recent Self-Assessment Examinations

OBJECTIVE

The purpose of this study is to determine the most commonly tested topics and the question taxonomy of the American Academy of Orthopaedic Surgeons Self-Assessment Examinations (SAE) from 2009 through 2014.

DESIGN

All SAEs were analyzed from 2009 through 2014. The SAEs were separated by subject and the questions of each SAE were analyzed for topic, taxonomic classification, and question type.

RESULTS

A total of 2107 questions were reviewed from 10 different subjects. In all, 6 of the 9 subjects had roughly 1/3 of their questions composed of the 3 most commonly tested topics. Each subject had at least 1 trauma-related question within its top 5 most commonly tested topics. Almost half (47%) of all questions were of taxonomy 1 classification and 29% were taxonomy 3. The Basic Science SAEs had the greatest percentage of taxonomy 1 questions of any subject (83%) whereas Trauma contained the highest percentage of taxonomy 3 questions (47%).

CONCLUSIONS

Certain topics within each subject are consistently tested more often than other topics. In general, the 3 most commonly tested topics comprise about one-third of total questions and orthopedic surgeons should be very familiar with these topics in order to best prepare for standardized examinations.

Evaluation of Pediatric Questions on the Orthopaedic In-Training Examination-An Update

BACKGROUND

Pediatric orthopaedics is tested frequently on the Orthopaedic In-Training Examination (OITE). The most recent data on the pediatrics section of the OITE were generated from content 10 years old. The purpose of this study is to assess the pediatric orthopaedic questions on the 2011 to 2014 OITE, and to compare question categories and cognitive taxonomy with previous data.

METHODS

Four years (2011 to 2014) of OITE questions, answers, and references were reviewed. The number of pediatric questions per year was recorded, as well as presence of a clinical photo or imaging modality. Each question was categorized and assigned a cognitive taxonomy level. Categories included: knowledge; knowledge-treatment modalities; diagnosis; diagnosis/recognition of associated conditions; diagnosis/further studies; and diagnosis/treatment. Cognitive taxonomy levels included: simple recall, interpretation of data, and advanced problem-solving.

RESULTS

The 3 most commonly covered topics were upper extremity trauma (17.4%), scoliosis (10.1%), and developmental dysplasia of the hip (5.7%). Compared with previous data, the percentage of pediatric questions was constant (13% vs. 14%). Categorically, the more recent OITE examinations contained significantly fewer questions testing simple knowledge (19% vs. 39%, P=0.0047), and significantly more questions testing knowledge of treatment modalities (17% vs. 9%, P=0.016) and diagnosis with associated conditions (19% vs. 9%, P=0.0034). Regarding cognitive taxonomy, there was a significant increase in the average number of questions that required advanced problem-solving (57% vs. 46%, P=0.048). Significantly more questions utilized clinical photographs and imaging studies (62% vs. 48%, P=0.012). The most common reference materials provided to support correct responses included Lovell and Winter's Pediatric Orthopaedics (25.7%) and the Journal of Pediatric Orthopaedics (23.4%).

CONCLUSIONS

Although the percentage of pediatric questions on the OITE has remained essentially constant, the percentage of questions requiring advanced problem-solving or interpretation of images has increased significantly in the past 10 years. Knowledge of question type and content may be helpful for those involved in resident education and in the development of didactic pediatric orthopaedic curricula.

LEVEL OF EVIDENCE

Level IV.

Levels of Evidence for Foot and Ankle Questions on the Orthopaedic In-Training Examination: 15-Year Trends

BACKGROUND

The foot and ankle (FA) content domain is a component of the orthopaedic in-training examination (OITE). Levels of evidence (LoE) have been infrequently studied on the OITE. The purpose of this study is to determine if LoE for primary journal articles referenced for FA questions increased over a 15-year period. We also aim to determine if reference characteristics and question taxonomy have changed during this period.

METHODS

All 132 questions and 261 references in the FA content domain from 1995 to 1997 and from 2010 to 2012 were included. We defined the characteristics of each reference and taxonomy of each question. Every primary journal article was assigned a LoE based on American Academy of Orthopaedic Surgeons (AAOS) guidelines.

RESULTS

Foot & Ankle International (FAI) was the most frequently cited journal. The change in the distribution of the Buckwalter classifications was statistically significant (p = 0.0286) with an increase in the number of clinical management questions. There were more level I studies on the 2010 to 2012 OITE (p = 0.0478) 6/54 (11%) of questions on the 2010 to 2012 OITE cited level I or II evidence compared with 3/78 (4%) on the 1995 to 1997 examinations (p = 0.1035).

CONCLUSIONS

There is a trend toward improved LoE for journal articles within the FA content domain on the OITE over a 15-year period, particularly when analyzing the increase in level I studies. FAI is the most frequently cited journal and questions increasingly test clinical management concepts.

CLINICAL RELEVANCE

Our results can be used to help improve resident self-study and suggest that reviewing recent FAI articles may aid OITE preparation.

LEVEL OF EVIDENCE

Basic Science.

Analysis of the Korean Orthopedic In-Training Examination: The Hip and Pelvis Section

Purpose

The purpose of this study was to analyze the questions in the hip and pelvis section of the Korean Orthopaedic In-Training Examination (KOITE).

Materials and Methods

We analyzed all KOITE questions pertaining to hip and pelvis surgery between 2010 and 2014. A thorough analysis of the contents was performed after categorizing as tested topics, imaging modalities used, taxonomic classification, and recommended references. We also analyzed the scores of the hip and pelvis section of the KOITE.

Results

Seventy-five of five-hundred questions (weight, 15.0%) were related to the hip and pelvis. Trauma including fracture and dislocation (26/75, 34.7%) was asked more commonly than disease and basics. The description-only questions (65/75, 86.7%) were the most frequently asked. According to taxonomic classification, taxonomy 3 (decision; 39/75, 52.0%) was most frequently asked. Campbell's Operative Orthopedics (52/75, 69.3%) was the reference that covered most of the questions.

Conclusion

This analysis of the hip and pelvis section of KOITE could be used for resident training programs in teaching hospitals.