Impact of a weekly reading program on orthopedic surgery residents' in-training examination

Video-based digital platforms as an educational resource for the surgical preparation of orthopedic surgeons

OBJECTIVE

The aim of the study was to research the video-based digital platforms that orthopedic specialists in Turkey use as an educational resource in their surgical preparations that they have not seen or done before, the frequency of their use of these platforms, and their trust in these platforms, with a survey study.

METHODS

The importance of video-based digital platforms in surgical preparations that surgeons have not seen or done before was measured using the data obtained from 181 orthopedic specialists using a survey prepared on an Internet-based server (docs.google.com).

RESULTS

Orthopedists used video-based digital platforms with a ratio of 38.7% among the educational resources in their surgical preparations that they have not seen or done before. There was no significant difference between the specialists with a surgical experience of 1-10 years and more than 10 years of experience in terms of using video-based digital platforms in surgical preparation (p>0.05). A total of 81.2% of the participants used only video-based digital platforms in the preparation of a surgical procedure they have never seen before. The most frequently used digital platform was YouTube, and 62% of the participants considered these platforms reliable.

CONCLUSION

Orthopedic specialists in Turkey primarily and frequently use video-based digital platforms as a training resource in their preparations for surgery that they have not seen or done before. The establishment or support of platforms with evidence-based content with references from official orthopedic institutions and organizations can increase the trust of orthopedic specialists in these platforms.

Does the Preferred Study Source Impact Orthopedic In-Training Examination Performance?

OBJECTIVE

This study examines the role of electronic learning platforms for medical knowledge acquisition in orthopedic surgery residency training. This study hypothesizes that all methods of medical knowledge acquisition will achieve similar levels of improvement in medical knowledge as measured by change in orthopedic in-training examination (OITE) percentile scores. Our secondary hypothesis is that residents will equally value all study resources for usefulness in acquisition of medical knowledge, preparation for the OITE, and preparation for surgical practice.

DESIGN

9 ACGME accredited orthopedic surgery programs participated with 95% survey completion rate. Survey ranked sources of medical knowledge acquisition and study habits for OITE preparation. Survey results were compared to OITE percentile rank scores.

PARTICIPANTS

386 orthopedic surgery residents SETTING: 9 ACGME accredited orthopaedic surgery residency programs RESULTS: 82% of participants were utilizing online learning resources (Orthobullets, ResStudy, or JBJS Clinical Classroom) as primary sources of learning. All primary resources showed a primary positive change in OITE score from 2018 to 2019. No specific primary source improved performance more than any other sources. JBJS clinical classroom rated highest for improved medical knowledge and becoming a better surgeon while journal reading was rated highest for OITE preparation. Orthopedic surgery residents' expectation for OITE performance on the 2019 examination was a statistically significant predictor of their change (decrease, stay the same, improve) in OITE percentile scores (p<0.001).

CONCLUSIONS

Our results showed that no specific preferred study source outperformed other sources. Significantly 82% of residents listed an online learning platform as their primary source which is a significant shift over the last decade. Further investigation into effectiveness of methodologies for electronic learning platforms in medical knowledge acquisition and in improving surgical competency is warranted.

Daily versus weekly evidence reports for orthopaedic surgeons in India: A mixed-methods study

BACKGROUND

There is a dearth of research regarding the impact of evidence-based medicine (EBM) tools, such as evidence summaries, in developing countries. The goals of this study were to: investigate accessibility, use, and impact of an online EBM knowledge dissemination portal in orthopaedic surgery in India; explore whether receiving daily targeted evidence summaries results in more frequent use of an EBM tool compared with receiving general weekly reports; and identify and explain the barriers and benefits of an online EBM resource in the Indian context.

METHODS

Forty-four orthopaedic surgeons in Pune, India, were provided free access to OrthoEvidence (OE), a for-profit, online EBM knowledge dissemination portal. Participants were subsequently randomized into 2 groups-1 group received daily targeted evidence summaries while the other received general weekly summaries. This study employed an explanatory sequential mixed methods design that incorporated 2 questionnaires, OE usage data, and semi-structured interviews to gain insight into the surgeons' usage, perceptions, and impact of OE.

RESULTS

There were no observable differences in OE usage between groups. OE was deemed to be comprehensive, practical, useful, and applicable to clinical practice by the majority of surgeons. The exit survey data revealed no differences between groups' perceptions of the OE tool. semi-structured interviews revealed barriers to keeping up with evidence that included limited access to relevant medical literature and limited incentive to keep up with current evidence.

CONCLUSIONS

Neither frequency of delivery (daily versus weekly) nor targeted versus general content affected the use of evidence summaries. Facilitating uptake of current evidence into clinical practice among Indian orthopedic surgeons may require additional components beyond dissemination of evidence summaries.

Orthopaedic In-Training Examination Resources and Residency Training for the Foot and Ankle Domain

BACKGROUND

Residency programs use the annual Orthopaedic In-Training Examination (OITE) prepared by the American Academy of Orthopaedic Surgeons (AAOS) to monitor resident progress and prepare them for the part 1 of the American Board of Orthopaedic Surgeons (ABOS) Certifying Examination. The purpose of this study was to determine resources residents currently use to prepare for the OITE and also to learn about their perception of training they receive in the foot and ankle subspecialty in their program and their interest in foot and ankle fellowship after residency.

METHODS

An anonymous survey was sent to both allopathic programs and osteopathic residents to learn what resources residents used to study for the OITE, preparatory question sets, on-call resources, their perception on training received in foot and ankle surgery, and their intent to pursue fellowship training.

RESULTS

A total of 130 residents participated in the survey. The majority of residents in allopathic and osteopathic residencies used Orthobullets (OB) to prepare for the OITE and use this resource while on-call. Most residents also used OB question sets to study along with the AAOS self-assessment examinations. In total, 43.2% of osteopathic residents felt they did not get enough exposure to foot and ankle subspecialty while in training, in contrast to 31.2% of allopathic residents. A total of 35% of all orthopaedic surgery residents felt they lacked enough exposure to foot and ankle orthopaedic surgery. Only 7 residents (6%, 6 allopathic, 1 osteopathic) intended to pursue a foot and ankle fellowship following graduation.

CONCLUSION

Online resources such as OB continue to be frequently used by residents for preparation for the OITE. Greater than one-third of orthopaedic residents feel they do not get enough exposure to foot and ankle orthopaedic surgery. Improvement in this area could be helped by continued endeavors from the American Orthopaedic Foot and Ankle Society such as the Visiting Professor Program and Resident Scholarship Program.

LEVELS OF EVIDENCE

Level V: Single Cross-Sectional Study.

The Importance of Determining Trainee Perspectives on Procedural Competencies During Spine Surgery Clinical Fellowship

STUDY DESIGN

Longitudinal survey.

OBJECTIVE

It remains important to align competence-based objectives for training as deemed important by clinical fellows to those of their fellowship supervisors and program educators. The primary aim of this study was to determine trainee views on the relative importance of specific procedural training competencies. Secondarily, we aimed to evaluate self-perceived confidence in procedural performance at the commencement and completion of fellowship.

METHODS

Questionnaires were administered to 68 clinical fellows enrolled in the AOSNA fellowship program during the 2015-2016 academic year. A Likert-type scale was used to quantify trainee perspectives on the relative importance of specific procedural competencies to their training base on an established curriculum including 53 general and 22 focused/advanced procedural competencies. We measured trainee self-perceived confidence in performing procedures at the commencement and completion of their program. Statistical analysis was performed on fellow demographic data and procedural responses.

RESULTS

Our initial survey response rate was 82% (56/68) and 69% (47/68) for the follow-up survey. Although most procedural competencies were regarded of high importance, we did identify several procedures of high importance yet low confidence among fellows (ie, upper cervical, thoracic discectomy surgery), which highlights an educational opportunity. Overall procedural confidence increased from an average Likert score of 4.2 (SD = 1.3) on the initial survey to 5.4 (SD = 0.8) by follow-up survey (P < .0001).

CONCLUSIONS

Understanding trainee goals for clinical fellowship remains important. Identification of areas of low procedural confidence and high importance to training experience will better guide fellowship programs and supervisors in the strategic delivery of the educational experience.

Development of a Case-based Reading Curriculum and Its Effect on Resident Reading

Textbook reading plays a foundational role in a resident's knowledge base. Many residency programs place residents on identical reading schedules, regardless of the clinical work or rotation the resident is doing. We sought to develop a reading curriculum that takes into account the clinical work a resident is doing so their reading curriculum corresponds with their clinical work. Preliminary data suggests an increased amount of resident reading and an increased interest in reading as a result of this change to their reading curriculum.

Residents and Program Director Perspectives Often Differ on Optimal Preparation Strategies and the Value of the Orthopedic In-Training Examination

OBJECTIVES

The purpose of this work was to compare resident and program director (PD) perspectives on the value of the Orthopaedic In-Training Examination (OITE), resident study habits, and best resources for optimal performance.

DESIGN

A national survey of orthopedic surgery residents and PDs.

SETTING

Mayo Clinic, Rochester, MN PARTICIPANTS: The survey was completed by 445 (41.5%) eligible orthopedic surgery residents and 37 (77.1%) PDs.

RESULTS

Although residents and PDs agreed on when (p = 0.896) and how much (p = 0.171) residents currently study, residents felt that the OITE was not as valuable of an assessment of their knowledge, and also felt their individual scores were less likely to remain confidential compared to PDs (p < 0.001). The mean OITE score below which residents were concerned about their ability to pass American Board of Orthopaedic Surgeons Part 1 was 9.7 percentile points higher than PDs threshold (42.3% vs. 32.6%, respectively, p = 0.003). Both groups agreed that it is important to dedicate focused study time to the OITE (p = 0.680) and to perform well (p = 0.099). Regarding the best resources and preparation strategies, both residents and PDs tended to agree on the value of most (6 of 10) study methods. Residents ranked practice question websites (mean ranking of 2.6 vs. 3.8 of 10, respectively; p < 0.001) and formal rotations in a subspecialty (6.0 vs. 7.7 respectively, p < 0.001) higher than PDs. In contrast, PDs tended to value their program's formal OITE prep program (4.1. vs. 5.3, respectively, p = 0.012) and reading primary literature (5.6 vs. 6.6, respectively, p = 0.012) more than residents.

CONCLUSION

Residents and PDs agreed on many critical components of this process; however, a number of key differences in perspectives exist.

Orthopedic In-Training Examination: A Performance Review Based on Program- and Resident-Specific Characteristics

OBJECTIVES

The orthopedic in-training examination (OITE) is the most common and objective method used to assess resident knowledge in the United States. As such, residents and programs use a number of strategies to maximize OITE performance. The purpose of this work was to better understand what strategies were being implemented and to determine which program-specific and resident-specific characteristics best correlate with improved scores.

DESIGN

A national survey of orthopedic residents and program directors (PDs) was performed to better understand OITE performance and correlate scores with various test preparation strategies.

SETTING

Mayo Clinic, Rochester, MN.

PARTICIPANTS

The survey was completed by 33 of 48 (68.8%) PDs and 341 of 878 (38.8%) eligible residents.

RESULTS

The most commonly used program-wide strategies were as follows: negative consequences for poor performance (72.7%), formal OITE prep program (54.5%), and purchase of OITE test prep material for residents (51.5%). The program-specific characteristics that had the strongest correlation with increased scores were negative consequences for poor performance (p < 0.001), high value placed on the OITE by PD and residents (p < 0.001), excusing residents from clinical duties the evening prior (p < 0.001), having residents take the examination on different days (p = 0.012), and allowing residents to lead a review course (p = 0.047). The resident-specific characteristics that correlated most with score were increased study time leading up to the test (p = 0.031) and attendance at their program's OITE prep program (p = 0.062).

CONCLUSIONS

Although programs and residents looking to improve knowledge acquisition and OITE scores use a number of techniques, a few distinct strategies correlate with the greatest increases in OITE performance. These may be appropriate methods to consider for those looking to improve their performance in coming years.

Orthopedic In-Training Examination Question Metrics and Resident Test Performance

First administered in November 1963, the orthopedic in-training examination (OITE) is now distributed to more than 4000 residents in over 20 countries and has become important for evaluation of resident fund of knowledge. Several studies have assessed the effect of didactic programs on resident performance, but only recently has it become possible to assess detailed testtaking metrics such as time spent per question. Here, we report the first assessment of resident OITE performance utilizing this full electronic dataset from two large academic institutions.

Full 2015 OITE score reports for all orthopedic surgery residents at two institutions were anonymized and compiled. For every question answered by each resident, the resident year, question content or domain, question result (correct or incorrect), and answer speed were recorded. Data were then analyzed to determine whether resident year, result, or domain affected answer speed and whether performance in each subspecialty domain varied based on resident year in training.

Data was available for 46 residents and 12,650 questions. Mean answer speed for questions answered correctly, 54.0±48.1 s, was significantly faster than for questions answered incorrectly, 72.2±61.2 s (P<0.00001). When considering both correct and incorrect answers, PGY-1s were slower than all other years (P<0.02). Residents spent a mean of nearly 80 seconds on foot and ankle and shoulder and elbow questions, compared to only 40 seconds on basic science questions (P<0.05).

In education, faster answer speed for questions is often considered a sign of mastery of the material and more confidence in the answer. Though faster answer speed was strongly associated with correct answers, this study demonstrates that answer speed is not reliably associated with resident year. While answer speed varies between domains, it is likely that the majority of this variation is due to question type as opposed to confidence. Nevertheless, it is possible that in domains with more tiered experience such as shoulder, answer speed correlates strongly with resident year and percentage correct.

Validation of a Web-Based Curriculum for Resident Education in Orthopedic Surgery

BACKGROUND

The Orthopedic In-Training Examination is an annual standardized examination with multiple-choice questions focused on application of orthopedic surgery core knowledge and principles. The outcome of this test can be used to both predict how residents are progressing in their orthopedic knowledge as well as assess their likelihood to pass the Orthopedic Board Examinations, that is the American Boards of Orthopedic Surgery Part 1, following completion of residency. Preparation for the examination can be difficult, as residents commonly have limited study time to review the vast amounts of available published literature. The objective of our study is to evaluate the effectiveness of the Orthopaedic In Training Examination (OITE) scores and the participants' perceived utility of the curriculum for OITE preparation.

METHODS

Residents from 5 US Orthopedic residencies (4 M.D. and 1 D.O.) were included in a pilot program of the Orthobullets PASS curriculum in the academic year 2013 to 2014. Only residents enrolled in the PASS curriculum who completed both the 2013 and 2014 OITEs were included in the final analysis (n = 71). We used the OITE 2013 and 2014 rank postgraduate year (RPGY) reported scores to assess for efficacy of the PASS curriculum, as the RPGY score provides postgraduate year-of-training matched analysis to control for expected increased levels of knowledge with subsequent retesting.

RESULTS

While OITE scores incrementally increased for the group as a whole (n = 71, RPGY mean improvement = +2.5%, p = 0.406), the junior resident subgroup (postgraduate year 1-2) produced a statistically significant increase in scores (n = 28, RPGY mean increase = 10.1%, p = 0.0260). Nearly 90% (38/42) of curriculum participants surveyed reported a preference to complete a similar review curriculum for future OITE preparation. The participants completing greater than 150 PASS questions (n = 57) were analyzed for OITE predictive capacity of the PASS curriculum. Pearson analysis with PASS questions percent answered correctly and 2014 OITE raw score (independent and dependent variables, respectively) suggests moderate correlation of the variables (R = 0.682, p < 0.05).

CONCLUSION

We propose that the Orthobullets PASS curriculum is a useful tool for OITE preparation, especially for junior residents, with a focus on achieving competency for a broad fund of orthopedic knowledge, whereas mastery of these topics would be better suited through using complementary sources.

The 50 Most Cited Articles in Rotator Cuff Repair Research

Analysis of the number of citations within a given specialty provides information on the classic publications of that specialty. The goals of this study were to identify the 50 most cited articles on rotator cuff repair and to analyze various characteristics of these articles. The ISI Web of Science (Thomson Reuters, Philadelphia, Pennsylvania) was used to conduct a search for the term rotator cuff repair. The 50 most cited articles were retrieved, and the following objective characteristics of each article were recorded: number of times cited, citation density, journal, country of origin, and language. The following subjective characteristics of each article were also recorded: article type (clinical vs basic science), article subtype, and level of evidence for clinical articles. Of the 50 most cited articles on rotator cuff repair, the number of citations ranged from 138 to 677 (mean, 232±133 citations) and citation density ranged from 3.8 to 53.5 citations per year (mean, 16.9±9.2 citations per year). The articles were published between 1974 and 2011, with most of the articles published in the 2000s (29 articles), followed by the 1990s (16 articles). The articles originated from 8 countries, with the United States accounting for 30 articles (60%). Overall, 66% of the articles were clinical and 34% were basic science. The most common article subtype was the clinical case series (48%). Of the 33 clinical articles, 24 (73%) were level IV. Among the 50 most cited articles on rotator cuff repair, the case series was the most common article subtype, showing the effect that publication of preliminary outcomes and new surgical techniques has had on surgeons performing rotator cuff repair. [Orthopedics. 2016; 39(6):e1045-e1051.].