Validity evidence for two objective structured clinical examination stations to evaluate core skills of the shoulder and knee assessment

Outcomes of inquiry-based learning in health professions education: a scoping review

Background

Open inquiry-based learning (IBL) that aims to foster higher-level thinking, is defined by students formulating their own questions and learning through exploration. The present study aimed to summarize the breadth of metrics used to evaluate health professions trainees in open IBL curricula.

Methods

We conducted a scoping review to identify publications detailing trainee outcomes in open IBL initiatives in health professions education. We queried five databases and included studies which described interventions with five phases of IBL (orientation, conceptualization, investigation, conclusion, and discussion). We completed abstract and full text reviews in duplicate. Data were collated and summarized.

Results

From 3030 record, 21 studies were included in the final extraction (k = 0.94), with nine involving physician trainees and twelve involving nursing trainees. Three studies used validated data collection tools to measure student inquiry behavior, and a single study used a validated data collection tool to measure critical thinking abilities. Most studies (n = 11) reported trainee self-reported satisfaction or perceived gain of skills as the primary outcome. All four studies using validated tools reported high scores in inquiry behaviors at the end of the curriculum and results on critical thinking skills were mixed. One study collected serial data, while remaining studies collected pre-post or post-only data.

Conclusion

IBL has the potential to cultivate a climate of curiosity among health professions learners. However, studies have relied heavily on subjective outcomes. Limited studies reported standardized measures of inquiry behaviors suggest favorable results. Curriculum innovations using IBL could make use of existing tools to better understand their impact on students' inquiry-oriented skills.

Impact of a Musculoskeletal "Mini-Residency" Professional Development Program on Knee Magnetic Resonance Imaging Orders by Primary Care Providers

BACKGROUND

The US Department of Veterans Affairs has created a portfolio of educational programs to train primary care providers (PCPs) in the evaluation and management of common musculoskeletal (MSK) conditions. Appropriate resource utilization for evaluation of knee pain, including limiting unnecessary magnetic resonance imaging (MRI) studies, is an important theme of these initiatives. The objective of this study was to report the utilization of knee MRI by PCP providers before and after the MSK education program and to determine the appropriateness of these MRI orders.

METHODS

Twenty-six PCPs participated in the MSK Mini-Residency educational program held in Salt Lake City between April 2012 and October 2014. Knee MRI orders submitted by these providers 12 months before and 12 months after their participation were reviewed. Magnetic resonance imaging orders were categorized as "inappropriate," "probably inappropriate," or "possibly appropriate," based on accepted guidelines for knee MRI utilization. Differences in the numbers of precourse and postcourse MRI orders for each of these categories were compared using Student t test.

RESULTS

Following our program, MRI orders decreased from 130 (precourse) to 93 (postcourse), a reduction of 28% ( p = 0.04). This reduction was observed entirely within the "inappropriate" and "probably inappropriate" categories; the number of orders categorized as "possibly appropriate" increased, but not significantly.

CONCLUSIONS

The MSK Mini-Residency training program was a successful educational intervention and was associated with a reduction in inappropriate knee MRI utilization for some participants, while keeping appropriate MRI utilization stable.

Difficulties with Learning Musculoskeletal Physical Examination Skills: Student Perspectives and General Lessons Learned for Curricular Design

Phenomenon: The development of foundational clinical skills, such as physical examination, is essential to becoming a competent clinician. Musculoskeletal medicine is often considered a specialized area of practice despite the high prevalence of musculoskeletal conditions in the general population and presenting to general clinical practices. Prior work has shown that medical learners and practicing clinicians have low confidence in these skills but understanding of the student perspective on why these skills are more difficult to acquire is unclear.Approach: Our study was guided by social constructivist learning theory to explore the learner experience and present their perspectives. Qualitative analysis investigated the difference between learning musculoskeletal physical examination versus other body systems, using the voices from 11 semi-structured focus group interviews. Participants included third-year medical students across two academic cohorts at one institution. Our analysis was grounded in the principles of phenomenology and used triangulation and reflexivity to provide rigorous analysis.Findings: Students provided rich and insightful perspectives regarding their experiences in learning musculoskeletal physical examination techniques. Four themes were developed from our data: a) the need for opportunities for both supervised and self-directed practice; b) assessment and competence as motivations for learning; c) the need for a different approach to the content and structure of musculoskeletal medicine and its associated examination techniques; and d) the need for distinct expertise and technical skill from musculoskeletal examination teachers.Insights: This study provides a valuable lens to critically reflect on existing curriculum and pedagogical approaches to musculoskeletal examination skills. Lessons from this study may be applicable to curriculum design in general, especially the teaching of physical examination skills, such as how it is taught and integrated with other content (including anatomy), how much practice is required, who teaches physical examination skills, and what faculty development is needed to standardize teaching. Promoting a learner-centered approach to the teaching and learning of these clinical skills will be beneficial to all stakeholders, especially to our future physicians and their patients.Supplemental data for this article is available online at https://doi.org/10.1080/10401334.2021.1954930 .

Using a station within an objective structured clinical examination to assess interprofessional competence performance among undergraduate nursing students

AIM/OBJECTIVE

To describe and analyse the use of a station within an OSCE to assess interprofessional competence performance in undergraduate nursing students. The specific objectives were: - To measure the students' level of competence performance in relation to the interprofessional competences Roles and Responsibilities, Communication and Teamwork. - To determine inter-observer concordance in the assessment of the interprofessional competences.

BACKGROUND

Teamwork competencies are key to improving patient safety and avoiding medical errors. Today, healthcare professionals work in interdisciplinary teams. To foster a culture of safety, some of the measures that can be taken at the individual, team and organisational levels include fostering clear communication among team members, knowledge of respective roles and functions, and deepening team functioning through respect and trust in judgement and capabilities. The World Health Organization recommends starting to develop these competencies in university studies, through interprofessional education. There are numerous programmes in universities all over the world, but more research is needed on the assessment of interprofessional education activities, preferably through objective methods. Competency performance can be assessed by an external evaluator, in a simulated environment, with the Objective Structured Clinical Examination, which is widely used in nursing.

DESIGN

Cross-sectional study.

METHODS

63 second-year nursing undergraduate students completed an interprofessional competencies station within an 8-station OSCE. Communication, Roles and Responsibility and Teamwork competences were assessed. The Interprofessional Collaborator Assessment Rubric (ICAR) was used as a model to assess the performance of students. Inter-observer concordance analysis was performed using the kappa coefficient and the concordance rate.

RESULTS

92.1% of students reached a good level in communication competence, 88.9% in roles and responsibility competence, and 55.6% in teamwork competence. The global concordance rate was 83.8%, and the kappa coefficient was 0.67.

CONCLUSIONS

Most students have demonstrated interprofessional competence performance at a good level. However, the inter-observer concordance obtained for some of the items was not as expected. The assessment of interprofessional competencies, as it deals mainly with relational and communicative aspects, requires greater preparation both in terms of the specification of assessment items and in agreement between examiners.

Lists of potential diagnoses that final-year medical students need to consider: a modified Delphi study

BACKGROUND

Contrastive learning is known to be effective in teaching medical students how to generate diagnostic hypotheses in clinical reasoning. However, there is no international consensus on lists of diagnostic considerations across different medical disciplines regarding the common signs and symptoms that should be learned as part of the undergraduate medical curriculum. In Japan, the national model core curriculum for undergraduate medical education was revised in 2016, and lists of potential diagnoses for 37 common signs, symptoms, and pathophysiology were introduced into the curriculum. This study aimed to validate the list of items based on expert consensus.

METHODS

The authors used a modified Delphi method to develop consensus among a panel of 23 expert physician-teachers in clinical reasoning from across Japan. The panel evaluated the items on a 5-point Likert scale, based on whether a disease should be hypothesized by final-year medical students considering given signs, symptoms, or pathophysiology. They also added other diseases that should be hypothesized. A positive consensus was defined as both a 75% rate of panel agreement and a mean of 4 or higher with a standard deviation of less than 1 on the 5-point scale. The study was conducted between September 2017 and March 2018.

RESULTS

This modified Delphi study identified 275 basic and 67 essential other than basic items corresponding to the potential diagnoses for 37 common signs, symptoms, and pathophysiology that Japanese medical students should master before graduation.

CONCLUSIONS

The lists developed in the study can be useful for teaching and learning how to generate initial hypotheses by encouraging students' contrastive learning. Although they were focused on the Japanese educational context, the lists and process of validation are generalizable to other countries for building national consensus on the content of medical education curricula.

Implementing an established musculoskeletal educational curriculum in a new context: a study of effectiveness and feasibility

Background: Musculoskeletal (MSK) problems are common, yet many primary care (PC) providers feel inadequately trained to manage these conditions. Previous studies describe successful MSK educational innovations at single sites, but none have reported on subsequent attempts to replicate or adapt these innovations to new contexts. This article presents a study of a national Veterans Affairs MSK training program modified to fit an existing PC educational program.Objectives: (1) To evaluate the effectiveness and feasibility of an adapted MSK curriculum in a new context. (2) To provide a model for adaptation studies in health professions education.Design: A national MSK shoulder and knee curriculum was adapted for San Francisco VA PC trainees, which included a small-group workshop and workplace learning within a newly-created MSK clinic. Effectiveness was evaluated by assessments of trainee confidence in exam and injection skills (via 5-point Likert scale) and faculty-observed performance of knee and shoulder exams (reported as percent of maximum possible score). Feasibility was evaluated by determining acceptability of the program to PC trainees (via 5-point Likert scale) and ability to implement the curriculum using local resources.Results: 52 trainees completed the training during a 2-year period. Trainees' confidence in MSK exam skills improved from 3.3 to 4.5 for shoulder, and from 3.5 to 4.6 for knee. Confidence performing joint injections improved from 2.6 to 4.2 (shoulder) and 2.5 to 4.5 (knee) (p < 0.001 for all). Observed performance improved markedly - from 50% to 92% for shoulder, and 57% to 90% for knee. Feasibility was evident in high acceptability (5.0 for MSK clinic, and 4.9 for workshops), and successful and sustained implementation.Conclusions: Adapting an established MSK curriculum to a new context was effective and feasible. This may serve as a more efficient model for improving trainee education than de novo curriculum design at individual sites.

Stress, anxiety, self-efficacy, and the meanings that physical therapy students attribute to their experience with an objective structured clinical examination

BACKGROUND

Excessive stress and anxiety can impair learning. The objective structured clinical examination (OSCE) is a valuable tool to assess and promote the acquisition of clinical skills. However, significant OSCE-related stress and anxiety are frequently reported. The aim of this study was to investigate the relationships between physiological stress, self-reported levels of anxiety due to an OSCE, self-efficacy, and the meanings that physical therapy students attribute to their experience with the exam.

DESIGN

Concurrent mixed methods study.

METHODS

A total of 32 students took part in this study. All were enrolled in the third semester of a 10-semester Physical Therapy Bachelor Program. Salivary cortisol levels, self-reported anxiety (State-Trait Anxiety Inventory, STAI) were measured before the OSCE. Exam scores and self-efficacy ratings were also recorded. Correlations between variables were tested with the Pearson correlation, with ɑ at 0.05. Semi-structured interviews were used to explore the personal perspectives of students. Thematic analysis was used to investigate emergent themes.

RESULTS

Trait anxiety scores were significantly higher than normative values (p < 0.001). A high proportion of students showed high (STAI> 49) state anxiety (37.5%) and trait anxiety (65.6%). Salivary cortisol was not associated anxiety (p > 0.05). Neither stress nor anxiety correlated with OSCE scores. A moderate and significant direct correlation was found for self-efficacy scores and OSCE scores (r = 0.475, p = 0.007). Students reported that confidence had a calming effect and led to better self-perceived performance. They also reported that the OSCE can provide meaningful learning experiences despite being stressful.

CONCLUSIONS

A high proportion of our students reported a stable/lingering negative affect. However, neither stress nor anxiety related to OSCE scores. Students' confidence in their capabilities was correlated with their performance. Their subjective reports suggest that self-confidence may have protected them from the negative effects of stress and anxiety on academic performance.

Deconstructing the Joint Examination: A Novel Approach to Teaching Introductory Musculoskeletal Physical Examination Skills for Medical Students

Introduction

Musculoskeletal (MSK) disorders are very common, but suboptimal teaching of MSK medicine occurs and expert clinicians agree that MSK physical examination (PE) skills can be confusing and complicated for medical students. An innovative approach in introductory teaching of MSK PE skills was developed using constructivist theory for second-year medical students.

Methods

We implemented the MSK PE curriculum innovation in the second year of a four-year MD program, utilizing a standard framework with spaced practice and clinician coaching. We evaluated this curriculum by comparing the innovation group (n = 123) to a historical control group (n = 134) using an anonymous survey and OSCE station scores. Data analysis included repeated measures analysis of variance comparing students' self-confidence in MSK PE to students' self-confidence in other systems-based PEs, as well as independent t-test comparisons of self-confidence scores and MSK-specific OSCE station scores between the historical and innovation groups.

Results

The mean self-assessed confidence of the historical group was significantly lower for the MSK PE than all other PEs (p < 0.001), except for the neurological PE. Significant improvement in MSK PE self-confidence was noted with the innovation group (t(259) = -4.05, p < 0.001). OSCE scores significantly improved in MSK-specific stations, with medium to large effect size across the different stations.

Discussion

We successfully used a framework of deconstruction, repetition, and spaced practice to develop fundamental MSK PE skills in preclerkship medical students. This curriculum structure provides an effective example for teaching introductory MSK PE skills to early medical learners.

SimLEARN Musculoskeletal Training for VHA Primary Care Providers and Health Professions Educators

A simulation-based training curricula applied to the primary care evaluation and management of shoulder and knee pain resulted in improved access to care for veterans and cost savings for the health care system.

Turning Objective Structured Clinical Examinations into Reality

Objective structured clinical examinations assess learners "showing how" to perform complex clinical tasks. Devised as summative evaluations, these examinations with immediate feedback are useful formative evaluations to improve learner performance. This review describes how objective structured clinical examinations have been used in rheumatology education. Steps for creating an objective structured clinical examination are discussed. Validity and reproducibility are important considerations, especially for high-stakes summative objective structured clinical examinations. Consideration of the potential benefits in clinical education and their hazards are reviewed. When well-designed, formative objective structured clinical examinations have high educational value for learners and medical educators.

Interprofessional Musculoskeletal Education: A Review of National Initiatives from the Department of Veterans Affairs

This article reviews several national programs in musculoskeletal education initiated by the Department of Veterans Affairs over the past decade. These programs have become sustained interprofessional opportunities for learners across disciplines and along the continuum of health professions education (HPE) and training pathways. This article also describes opportunities for leaders in rheumatology and other HPE programs to join these efforts and to collaborate in the scholarship that will be necessary in constructing educational programs fit for the purpose of ensuring a well-trained, competent workforce of health care providers.

Long-Term Retention of Musculoskeletal Ultrasound Training During Residency

The Accreditation Council for Graduate Medical Education and the American Board of Physical Medicine and Rehabilitation developed milestones for evaluation of resident physicians that include proper musculoskeletal ultrasound examination of major joints. To date, there have been no published data demonstrating acquisition and retention of these skills and correlation with the milestone evaluation. The investigators developed and implemented a curriculum in musculoskeletal ultrasound examination for Physical Medicine and Rehabilitation residents at a large academic medical center. The investigators chose the following six joints for training and evaluation: ankle, elbow, hip, knee, shoulder, and wrist/hand. The program included: (1) didactic lectures on anatomy and ultrasound technique; (2) peer-led demonstrations of the procedure on a standardized patient (SP); (3) individual practice on standardized patients; (4) faculty observation and feedback; (5) review sessions and additional practice; and (6) assessment of skills in an objective structured clinical examination. From 2013 to 2017, 30 physical medicine and rehabilitation residents were trained and evaluated. The results, based on objective structured clinical examination scores, showed that most residents achieved the appropriate level of competency for their year. A blended, standardized curriculum in musculoskeletal ultrasound instruction with assessment by an objective structured clinical examination can be used to evaluate musculoskeletal ultrasound skills and can help align this education with residency milestones.

The impact of a musculoskeletal training program on residents' recognition and treatment of osteoporosis

BACKGROUND

Osteoporosis is inadequately treated in primary care settings. Under-recognition of the condition among male Veterans may contribute to this problem. In order to improve understanding of bone health in older male patients, we developed the "Musculoskeletal (MSK) Education Week", a multidisciplinary clinical training initiative within a primary care ambulatory rotation for internal medicine (IM) residents at the Salt Lake City VA Medical Center. The objective of this study was to evaluate the impact of this program on trainees' recognition of osteoporosis or treatment of this condition following the training experience.

METHODS

We examined several clinical behaviors of post-graduate year 1 (PGY-1) IM trainees following their participation in the MSK Education Week between July 1-April 30, 2014. To determine the prevalence of these clinical behaviors, we conducted an observational study of patients age 50 and older enrolled at the Salt Lake City VA Healthcare System from July 1, 2013 to May 31, 2014. We used time-dependent multivariable Cox proportional hazard models to evaluate the impact of the training program on 4 osteoporosis-related outcomes: (1) completion of dual energy X-ray absorptiometry (DXA) scan, (2) diagnosis of osteopenia, (3) diagnosis of osteoporosis, and (4) initiation of osteoporosis medications.

RESULTS

Twenty-six PGY-1 IM residents participated in the MSK Education Week, and 43,678 Veterans were identified over these periods of observation. In the Veterans cohort, 1154 had an encounter with a provider who had completed the training (and were therefore "exposed" to the training) and 42,524 Veterans did not. After adjusting for confounders, the effect of the provider training program was significant for DXA (HR = 1.78, 95% CI: 1.11, 2.87), osteoporosis diagnosis (HR = 3.90, 95% CI: 2.09, 7.29), and initiation of medications (HR = 2.87, 95% CI: 2.02, 4.09) outcomes.

CONCLUSIONS

We have shown that IM residents' participation in the MSK Education Week was associated with significantly improvements in their completion of DXA scans, diagnosis of osteoporosis, and initiation of fracture-reducing medications in a population of US Veterans. Long-term follow up is needed to determine whether these initial results are followed by actual reductions in osteoporotic fractures.

"Mini-Residency" in Musculoskeletal Care: a National Continuing Professional Development Program for Primary Care Providers

INTRODUCTION

A cost-effective professional development program enhancing musculoskeletal (MSK) skills of physicians and allied health providers working in primary care settings has been reported at a single site. This article describes the first 2 years of the national expansion and implementation of a 3-day "MSK Mini-residency."

METHODS

Faculty from Veterans Affairs (VA) medical centers worked in partnership with national program faculty from the Salt Lake City VA to present an intensive, integrated, multidisciplinary program to strengthen the skills of primary care providers in evaluating and managing MSK conditions common in primary care. Course assessments included written surveys and a two-station observed structured clinical examination (OSCE) evaluating the physical examination of the shoulder and knee.

RESULTS

In the first 2 years of the program, 13 VA facilities participated. Two hundred twenty-seven health care providers, including 135 physicians, were trained. Two hundred seven participants (91 %) completed all pre- and post-course written assessments and the two-station OSCE.

DISCUSSION

The MSK Mini-residency program is an effective and well-received mixed-method educational initiative to strengthen the skills of primary care physicians and other health care providers in evaluating and managing patients with MSK complaints and to document their competence in performing physical examinations of the shoulder and knee. The 2-year experience in implementation suggests that this model of educational partnerships is a feasible approach to disseminating innovative educational programs in a way that preserves curricular consistency yet is adaptable to local needs.

Effectiveness of an Interprofessional and Multidisciplinary Musculoskeletal Training Program

BACKGROUND

Musculoskeletal (MSK) problems are common, and a recent US Bone and Joint Initiative calls for new models of education and professional collaboration. Evidence of feasibility and acceptability of innovative methods are needed.

OBJECTIVE

We assessed if an experimental immersion interdisciplinary MSK curriculum would be acceptable to residents from different specialties, be feasible within existing rotations, and be effective in strengthening clinical skills.

METHODS

Through funding from the Veterans Affairs Office of Academic Affiliations and the Office of Specialty Care, we developed a Center of Excellence in MSK Care and Education. A core element is the monthly MSK Education Week, which teaches skills and provides opportunities to apply these in clinical settings. Participants include internal medicine, physical medicine and rehabilitation, and orthopaedic surgery residents, as well as students and residents from other health professions programs. All were assigned to the MSK week in lieu of other clinical experiences. Faculty encompassed primary care, rheumatology, endocrinology, orthopaedics, and physical medicine and rehabilitation. Assessments include surveys and a 2-station objective structured clinical examination (OSCE).

RESULTS

Since 2012, a total of 176 trainees have participated. Percentage of trainees reporting ability to evaluate and manage MSK complaints increased (9% to 87% for shoulder; 18% to 86% for knee), and confidence performing MSK injections increased from 10% to 70%. Competency in evaluation of shoulder and knee pain was confirmed by OSCEs.

CONCLUSIONS

The MSK week program was accepted by residents from the 3 specialties, with learners reporting improved ability to perform shoulder and knee examinations, as demonstrated by OSCEs.