An analysis of the Orthopaedic In-Training Examination rehabilitation section

The Evolution of the Musculoskeletal Trauma Section of the Orthopaedic In-Training Examination

INTRODUCTION

In 1963, the American Academy of Orthopaedic Surgeons administered the Orthopaedic In-Training Examination (OITE), the first and longest running yearly medical specialty examination. There have been no recent studies to evaluate the content of the musculoskeletal trauma section of the OITE.

METHODS

We analyzed all questions that were classified by the American Academy of Orthopaedic Surgeons as musculoskeletal trauma from 2012 to 2019. We recorded the number of musculoskeletal trauma questions in each examination, the topics and imaging modalities tested, the references cited, and the taxonomy classification of each question. We extrapolated from a similar musculoskeletal trauma study published in 2011 to create the previous examination cohort for comparison.

RESULTS

For the current cohort, the average number of musculoskeletal trauma questions was 43.5 questions per examination (18.4%). The most frequently tested topics were proximal tibia fractures, pediatric trauma, hip fractures, and diaphyseal femur fractures, respectively. In previous examinations, questions from T1 and 2 were tested significantly more frequently compared with the current examinations (P < 0.001 and P = 0.02, respectively). In the current cohort, T3 questions were tested significantly more frequently than previous examinations (P = 0.001). Previous examinations had significantly more questions without an image (36 questions per year versus 25 questions per year, P < 0.001). In current versions of the examination, radiographs are tested significantly more frequently than other imaging modalities (P < 0.001).

DISCUSSION

The musculoskeletal trauma section of the OITE has evolved. To improve and focus study efforts, residents may use this study as a guide when preparing for the examination.

Analysis of Questions in Sections of the Orthopaedic In-Training Examination: A Scoping Review

PURPOSE

The purpose of this review was to assess all available studies that analyzed the types of questions in individual sections of the Orthopaedic In-Training Examination, which may be used as a reference for residents studying for their examination.

METHODS

Following the Providing Innovative Service Models and Assessment extension for Scoping Reviews guidelines, a systematic review was conducted on studies that report on sections or question categories of the Orthopaedic In-Training Examination using PubMed, MEDLINE, and Web of Science databases. Two reviewers and an arbitrator reviewed and extracted relevant data from 20 included studies which made up the systematic review.

RESULTS

All 20 studies in the review reported the mean number of questions per section, with the highest coming from musculoskeletal trauma (18.9% to 19.0%). 18 studies reported the Buckwalter taxonomic classification; 42.0% of questions were T1, 18.2% were T2, and 39.5% were T3 with a wide range from section to section. Primary sources were nearly three times more likely to be cited when compared with textbook sources. There were 12 journals that were commonly cited with the most being the Journal of Bone and Joint Surgery: American Volume (17/18).

DISCUSSION

This study accurately portrays the characteristics of each section of the Orthopaedic In-Training Examination over the past 10 years. These data suggest that orthopaedic residents may be inclined to focus on musculoskeletal trauma, topics related to clinical management, and primary journal sources for studying. In addition, residency programs may choose to focus on higher yield sources or material to prepare their residents for the examination.

An updated analysis of shoulder and elbow questions on the Orthopedic In-Training Examination

BACKGROUND

The Orthopedic In-Training Examination (OITE) is an annual examination for orthopedic surgery residents used to assess orthopedic knowledge across a national standard. Having an updated understanding of currently tested topics and resources is useful to help residents guide their education.

PURPOSE

The purpose of this study is to analyze the shoulder and elbow domain of the OITE in an effort to provide current trends and commonly tested topics.

METHODS

All OITE questions related to shoulder and elbow topics over the years 2009-2013 and 2017-2020 were analyzed. Subcategories, the number and types of references used, publication lag time, imaging modalities, taxonomic classification, and resident performance were recorded.

RESULTS

Shoulder and elbow topics comprised 8.61% of all OITE questions from 2009-2013 and 2017-2020. The most commonly tested shoulder topics were rotator cuff arthropathy and reverse total shoulder arthroplasty (13.6%), followed by hemiarthroplasty and total shoulder arthroplasty (12.9%), rotator cuff-related pathology (12.9%), anterior shoulder instability and/or dislocation (10.2%), and general anatomy (10.2%). The most commonly tested elbow topics were trauma (21%), ulnar collateral ligament injuries (12.12%), general anatomy (10%), and arthroplasty (10%). Decisions regarding management or appropriate next steps (taxonomy T3) comprised 39% of all question types. The incorporation of imaging modalities continues to be an important component of the OITE questions. The Journal of Shoulder and Elbow Surgery (JSES), the Journal of the American Academy of Orthopedic Surgeons (JAAOS), the Journal of Bone and Joint Surgery (JBJS), and the American Journal of Sports Medicine (AJSM) comprised 65% of articles referenced in all questions over our analysis period.

CONCLUSION

This study provides an updated analysis of trends within the shoulder and elbow domain of the OITE. Application of these data can aid residents in their preparation for the examination.

Analysis of Hip and Knee Reconstruction Questions on the Orthopedic In-Training Examination

BACKGROUND

It is vital for orthopedic residents and residency programs to have a current understanding of the materials and resources utilized on the Orthopedic In-Training Examination (OITE) to tailor resident educational curricula accordingly. This study presents an updated analysis of the hip and knee section of the OITE.

METHODS

All OITE questions related to hip and knee reconstruction over six examinations between 2014 and 2019 were analyzed for topic, subtopic, taxonomy, imaging modalities, resident performance, and references.

RESULTS

There were 166 hip and knee reconstruction questions of 1600 OITE questions (10.4%) over a six-year period. The most commonly tested topics include mechanical properties of total knee and hip implants (10.8%), instability after THA (10.8%), periprosthetic fracture (10.2%), and prosthetic joint infection (10.2%). A total of 362 references were cited from 68 different sources. The most common sources were JOA, JBJS, JAAOS, and CORR, which were collectively responsible for 68% of all citations. There was an average publication lag of 7.1 years, with 75% of all citations falling within 10 years of the question date. Compared with a prior analysis from 2005 and 2009, there were significantly more complex multistep questions regarding treatment and fewer one-step knowledge recall questions (P = .003). Similarly, recent tests had significantly more questions involving interpretation of radiographs (55%, P < .001) and advanced imaging (9.6%, P < .001), compared with a decade ago.

CONCLUSIONS

The OITE continues to evolve over time, incorporating recent literature and topics. The current analysis identifies high-yield topics and resources that can guide resident preparation for the OITE hip and knee section.

The Changing Nature of the Oncology Section of the Orthopaedic In-Training Examination

INTRODUCTION

The Orthopaedic In-Training Examination (OITE) assesses orthopaedic resident knowledge over 275 multiple-choice questions.Since the first publication examining the contents of the pathology section was published over ten years ago, the pathology content has been renamed (oncology) and revamped. As the overall extent of these alterations is currently unknown, the efficacy of current orthopaedic oncology educational practices for optimal OITE performance should be questioned. To determine how the oncology (pathology) material has changed, we compared the following characteristics from previous examinations (2002 to 2006) to current examinations (2012 to 2016): (1) What are the average number of oncology questions being asked? (2) What are the specific imaging modalities presented for examinee interpretation? (3) Which pathologic diagnoses are commonly examined? (4) What is the pattern of taxonomic question classifications?

METHODS

The 2012 to 2016 OITE study guides were reviewed, and each oncology question was categorized into one of the following: benign or malignant, imaging modality grouping, common pathologic diagnosis, question type, and taxonomic classification. The aforementioned information was extrapolated from the previous pathology publication published in 2010 to create the previous examination cohort (2002 to 2006). The current examination characteristics were then compared with those of the previous examinations.

RESULTS

The current number of oncology OITE questions significantly decreased from previous years (27.2 versus 21.2; P = 0.015). Current examinations displayed a significant increase in testing the interpretation of diagnostic imaging modalities compared with previous examinations (78.3% versus 55.8%; P < 0.001). The current examinations examined a wide spectrum of pathologic diagnoses, including previously untested pathologies. The number of taxonomy 1 questions on current examinations significantly decreased (36.8% versus 24.5%; P = 0.032), whereas the number of taxonomy 3 questions significantly increased from previous examinations (48.1% versus 32.4%; P = 0.032).

DISCUSSION

This study demonstrated that the nature of the orthopaedic oncology (pathology) section has changed over the past 10 years. Although the overall number of pathology-related questions decreased, the difficulty level of these questions increased, demanding a higher level of knowledge and critical thinking. A formal orthopaedic oncology rotation may be the best method to educate and improve OITE oncology performance.

LEVEL OF EVIDENCE

Prognostic study, level III.

Orthopedic In-Training Examination: An Analysis of the Sports Medicine Section-An Update

OBJECTIVE

Previously published studies have evaluated Orthopedic In-Training Examination sports medicine questions, but none have evaluated whether question difficulty has changed over time.

DESIGN

Sports medicine subsection questions between 2012 and 2016 were evaluated and compared with previously published data on Orthopedic In-Training Examination from 2004 to 2009. Question categories were classified into 1 of 3 taxonomy levels-basic recall, diagnosis, and advanced problem solving.

SETTING

Medical University of South Carolina; Charleston, SC, 29425; Institution.

PARTICIPANTS

Two residents evaluated the Sports Subsection questions separately. Then an attending physician evaluated the questions to resolve discrepancies. A statistician was used for analysis.

RESULTS

Utilization of imaging modalities averaged 37%, with 28% (11/39) of the questions containing 2 or more imaging modalities. There were increases in utilization of advanced problem-solving questions (45% vs. 27%, p = 0.002) and decreases in basic recall questions (49% vs. 67%, p = 0.008) compared with previously published data.

CONCLUSIONS

While the percentage of the Orthopedic In-Training Examination represented by sports medicine has remained relatively unchanged, there were fewer questions requiring residents to demonstrate simple recall and diagnosis, and increased demand to perform advanced problem solving while utilizing multiple imaging modalities.

An Analysis of the Most Commonly Tested Topics and Their Taxonomy From Recent Self-Assessment Examinations

OBJECTIVE

The purpose of this study is to determine the most commonly tested topics and the question taxonomy of the American Academy of Orthopaedic Surgeons Self-Assessment Examinations (SAE) from 2009 through 2014.

DESIGN

All SAEs were analyzed from 2009 through 2014. The SAEs were separated by subject and the questions of each SAE were analyzed for topic, taxonomic classification, and question type.

RESULTS

A total of 2107 questions were reviewed from 10 different subjects. In all, 6 of the 9 subjects had roughly 1/3 of their questions composed of the 3 most commonly tested topics. Each subject had at least 1 trauma-related question within its top 5 most commonly tested topics. Almost half (47%) of all questions were of taxonomy 1 classification and 29% were taxonomy 3. The Basic Science SAEs had the greatest percentage of taxonomy 1 questions of any subject (83%) whereas Trauma contained the highest percentage of taxonomy 3 questions (47%).

CONCLUSIONS

Certain topics within each subject are consistently tested more often than other topics. In general, the 3 most commonly tested topics comprise about one-third of total questions and orthopedic surgeons should be very familiar with these topics in order to best prepare for standardized examinations.

Orthopedic In-Training Examination Question Metrics and Resident Test Performance

First administered in November 1963, the orthopedic in-training examination (OITE) is now distributed to more than 4000 residents in over 20 countries and has become important for evaluation of resident fund of knowledge. Several studies have assessed the effect of didactic programs on resident performance, but only recently has it become possible to assess detailed testtaking metrics such as time spent per question. Here, we report the first assessment of resident OITE performance utilizing this full electronic dataset from two large academic institutions.

Full 2015 OITE score reports for all orthopedic surgery residents at two institutions were anonymized and compiled. For every question answered by each resident, the resident year, question content or domain, question result (correct or incorrect), and answer speed were recorded. Data were then analyzed to determine whether resident year, result, or domain affected answer speed and whether performance in each subspecialty domain varied based on resident year in training.

Data was available for 46 residents and 12,650 questions. Mean answer speed for questions answered correctly, 54.0±48.1 s, was significantly faster than for questions answered incorrectly, 72.2±61.2 s (P<0.00001). When considering both correct and incorrect answers, PGY-1s were slower than all other years (P<0.02). Residents spent a mean of nearly 80 seconds on foot and ankle and shoulder and elbow questions, compared to only 40 seconds on basic science questions (P<0.05).

In education, faster answer speed for questions is often considered a sign of mastery of the material and more confidence in the answer. Though faster answer speed was strongly associated with correct answers, this study demonstrates that answer speed is not reliably associated with resident year. While answer speed varies between domains, it is likely that the majority of this variation is due to question type as opposed to confidence. Nevertheless, it is possible that in domains with more tiered experience such as shoulder, answer speed correlates strongly with resident year and percentage correct.