Evaluation of a problem-based learning workshop using pre- and post-test objective structured clinical examinations and standardized patients

Exploring the Study of Simulation as a Continuing Professional Development Strategy for Physicians

INTRODUCTION

Practicing physicians have the responsibility to engage in lifelong learning. Although simulation is an effective experiential educational strategy, physicians seldom select it for continuing professional development (CPD) for reasons that are poorly understood. The objective of this study was to explore existing evidence on simulation-based CPD and the factors influencing physicians' engagement in simulation-based CPD.

METHODS

A scoping review of the literature on simulation-based CPD included MEDLINE, Embase, and CINAHL databases. Studies involving the use of simulation for practicing physicians' CPD were included. Information related to motivations for participating in simulation-based CPD, study objectives, research question(s), rationale(s), reasons for using simulation, and simulation features was abstracted.

RESULTS

The search yielded 8609 articles, with 6906 articles undergoing title and abstract screening after duplicate removal. Six hundred sixty-one articles underwent full-text screening. Two hundred twenty-five studies (1993-2021) were reviewed for data abstraction. Only four studies explored physicians' motivations directly, while 31 studies described incentives or strategies used to enroll physicians in studies on simulation-based CPD. Most studies focused on leveraging or demonstrating the utility of simulation for CPD. Limited evidence suggests that psychological safety, direct relevance to clinical practice, and familiarity with simulation may promote future engagement.

DISCUSSION

Although simulation is an effective experiential educational method, factors explaining its uptake by physicians as a CPD strategy are unclear. Additional evidence of simulation effectiveness may fail to convince physicians to participate in simulation-based CPD unless personal, social, educational, or contextual factors that shape physicians' motivations and choices to engage in simulation-based CPD are explored.

Situational awareness within objective structured clinical examination stations in undergraduate medical training - a literature search

BACKGROUND

Medical students may not be able to identify the essential elements of situational awareness (SA) necessary for clinical reasoning. Recent studies suggest that students have little insight into cognitive processing and SA in clinical scenarios. Objective Structured Clinical Examinations (OSCEs) could be used to assess certain elements of situational awareness. The purpose of this paper is to review the literature with a view to identifying whether levels of SA based on Endsley's model can be assessed utilising OSCEs during undergraduate medical training.

METHODS

A systematic search was performed pertaining to SA and OSCEs, to identify studies published between January 1975 (first paper describing an OSCE) and February 2017, in peer reviewed international journals published in English. PUBMED, EMBASE, PsycINFO Ovid and SCOPUS were searched for papers that described the assessment of SA using OSCEs among undergraduate medical students. Key search terms included "objective structured clinical examination", "objective structured clinical assessment" or "OSCE" and "non-technical skills", "sense-making", "clinical reasoning", "perception", "comprehension", "projection", "situation awareness", "situational awareness" and "situation assessment". Boolean operators (AND, OR) were used as conjunctions to narrow the search strategy, resulting in the limitation of papers relevant to the research interest. Areas of interest were elements of SA that can be assessed by these examinations.

RESULTS

The initial search of the literature retrieved 1127 publications. Upon removal of duplicates and papers relating to nursing, paramedical disciplines, pharmacy and veterinary education by title, abstract or full text, 11 articles were eligible for inclusion as related to the assessment of elements of SA in undergraduate medical students.

DISCUSSION

Review of the literature suggests that whole-task OSCEs enable the evaluation of SA associated with clinical reasoning skills. If they address the levels of SA, these OSCEs can provide supportive feedback and strengthen educational measures associated with higher diagnostic accuracy and reasoning abilities.

CONCLUSION

Based on the findings, the early exposure of medical students to SA is recommended, utilising OSCEs to evaluate and facilitate SA in dynamic environments.

An objective structured clinical examination to assess problem-based learning

OBJECTIVES

To compare pharmacy students' performance on an objective structured clinical examination (OSCE) to their performance on a written examination for the assessment of problem-based learning (PBL); and to determine students' and faculty members' perceptions of OSCEs for PBL evaluations.

DESIGN

Four OSCEs were added to the written examination to assess 4 PBL cases in a third-year pharmacotherapy course. OSCE scores were compared to written examination scores. Faculty members evaluated student performance.

ASSESSMENT

OSCE performance did not correlate with the written-examination scores. Most students (≥ 75%) agreed that OSCEs reflected their learning from PBL and measured knowledge, communication, and clinical skills. A majority of faculty members (≥75%) agreed that OSCEs should be part of PBL assessment.

CONCLUSIONS

Addition of an OSCE to written examinations was valued and provided a more comprehensive assessment of the PBL experience.

Problem-based learning in postgraduate dental education: a qualitative evaluation of students' experience of an orthodontic problem-based postgraduate programme

INTRODUCTION

Problem based learning (PBL) has gained wide acceptance in undergraduate education, but less so in postgraduate education. Qualitative research methodologies can help us gain insight and understanding of the students' experience in undertaking such programmes.

AIM

To evaluate the experiences of postgraduate students undertaking a PBL based postgraduate programme.

METHODS

Focus groups and semi-structured interviews were used to investigate students' experiences. The qualitative data were coded, the codes were grouped into categories and a coding framework developed. From this a construct is proposed to give insight into students' experiences. The validity of this construct was examined by reviewing the literature.

RESULTS

Students entered the programme with high expectations, but soon found they had challenges to overcome. They came to realise that they were on a journey, rather than undertaking a sequence of separate individual learning events. PBL led to significant tensions both within the individuals and the group, caused by the conflict between appreciating PBL as a 'good thing' and yet finding that ;it just doesn't seem to work'. The fear of failure in a high stakes setting only exacerbated these tensions. The literature review revealed an inconsistent approach to the use of PBL and its evaluation in a postgraduate/continuing medical education setting. However it did provide evidence of similar findings in qualitative studies, supporting the construct developed.

CONCLUSION

We have proposed a construct to help in understanding the PBL experience for students undergoing a 'high stakes' orthodontic postgraduate programme with a core of PBL.

Assessing intern core competencies with an objective structured clinical examination

BACKGROUND

Residents are evaluated using Accreditation Council for Graduate Medical Education (ACGME) core competencies. An Objective Structured Clinical Examination (OSCE) is a potential evaluation tool to measure these competencies and provide outcome data.

OBJECTIVE

Create an OSCE to evaluate and demonstrate improvement in intern core competencies of patient care, medical knowledge, practice-based learning and improvement, interpersonal and communication skills, professionalism, and systems-based practice before and after internship.

METHODS

From 2006 to 2008, 106 interns from 10 medical specialties were evaluated with a preinternship and postinternship OSCE at Madigan Army Medical Center. The OSCE included eight 12-minute stations that collectively evaluated the 6 ACGME core competencies using human patient simulators, standardized patients, and clinical scenarios. Interns were scored using objective and subjective criteria, with a maximum score of 100 for each competency. Stations included death notification, abdominal pain, transfusion consent, suture skills, wellness history, chest pain, altered mental status, and computer literature search. These stations were chosen by specialty program directors, created with input from board-certified specialists, and were peer reviewed.

RESULTS

All OSCE testing on the 106 interns (ages 25 to 44 [average, 28.6]; 70 [66%] men; 65 [58%] allopathic medical school graduates) resulted in statistically significant improvement in all ACGME core competencies: patient care (71.9% to 80.0%, P < .001), medical knowledge (59.6% to 78.6%, P < .001), practice-based learning and improvement (45.2% to 63.0%, P < .001), interpersonal and communication skills (77.5% to 83.1%, P < .001), professionalism (74.8% to 85.1%, P < .001), and systems-based practice (56.6% to 76.5%, P < .001).

CONCLUSION

An OSCE during internship can evaluate incoming baseline ACGME core competencies and test for interval improvement. The OSCE is a valuable assessment tool to provide outcome measures on resident competency performance and evaluate program effectiveness.

Improving the quality of clinical care for patients with osteoporosis

Osteoporosis, leading to hip fractures and other fragility fractures, is prevalent in countries with a high life expectancy. Fractures have a high associated societal cost and disease burden. As a result, many countries have established screening guidelines for osteoporosis, especially targeting those at a higher risk. However, primary- and secondary-fracture prevention is currently suboptimal despite existence of effective medications. In addition, adherence to therapy is low. This special report outlines some of these challenges and evaluates different techniques for improving the quality of clinical care for patients with osteoporosis.

Continuing medical education, continuing professional development, and knowledge translation: improving care of older patients by practicing physicians

Many community-based internists and family physicians lack familiarity with geriatrics knowledge and best practices, but they face overwhelming fiscal and time barriers to expanding their skills and improving their behavior in the care of older people. Traditional lecture-and-slide-show continuing medical education (CME) programs have been shown to be relatively ineffective in changing this target group's practice. The challenge for geriatrics educators, then, is to devise CME programs that are highly accessible to practicing physicians, that will have an immediate and significant effect on practitioners' behavior, and that are financially viable. Studies of CME have shown that the most effective programs for knowledge translation in these circumstances involve what is known as active-mode learning, which relies on interactive, targeted, and multifaceted techniques. A systematic literature review, supplemented by structured interviews, was performed to inventory active-mode learning techniques for geriatrics knowledge and skills in the United States. Thirteen published articles met the criteria, and leaders of 28 active-mode CME programs were interviewed. This systematic review indicates that there is a substantial experience in geriatrics training for community-based physicians, much of which is unpublished and incompletely evaluated. It appears that the most effective methods to change behaviors involved multiple educational efforts such as written materials or toolkits combined with feedback and strong communication channels between instructors and learners.

Web-based learning versus standardized patients for teaching clinical diagnosis: a randomized, controlled, crossover trial

BACKGROUND

Little evidence exists to guide the selection of methods for teaching clinical diagnosis.

PURPOSE

To compare the efficacy, student preference, and cost of a Web-based (WB) program versus a standardized patient (SP) encounter for teaching clinical diagnosis skills to 2nd-year medical students.

METHODS

Randomized, controlled, crossover study comparing WB versus SP-based teaching for the clinical diagnosis of abdominal pain and headache. Outcome measures were performance on a 2-case SP examination (scored on the basis of a checklist completed by a faculty observer and an objective score on a postencounter subjective-objective assessment plan [SOAP] note), format preferences as assessed by end-of-course evaluations, and cost.

RESULTS

Thirty students consented to participate. WB and SP training produced similar scores on both the Abdominal Pain checklist (66% vs. 62%; p = .17) and Headache checklist (56% vs. 63%; p = .07). WB training produced a higher score on the Abdominal Pain SOAP note (69% vs. 47%; p = .006), but not the Headache SOAP note (69% vs. 67%; p = .85). Students rated the SP format higher than the WB format on all 7 preference measures. Start-up costs were estimated at 2,190 dollars for the SP format and 2,250 dollars for the WB format. Ongoing costs per case per student were estimated to be 45 dollars for the SP format and 30 dollars for the WB format.

CONCLUSIONS

WB and SP learning outcomes were comparable, but students preferred the SP format. Start-up costs were comparable, but the ongoing costs of the WB format were less expensive, suggesting that WB teaching may be a viable strategy.

Fielding incognito standardised patients as 'known' patients in a controlled trial in general practice

BACKGROUND

Incognito standardised patients (SPs) have only been used to represent new patients so far. The few trials with incognito SPs provide little detail on the method used for fielding them.

OBJECTIVE

To establish the feasibility of introducing SPs as 'known' patients (i.e. patients who have previously visited the practice) into practices, to indicate the required practice preparations, and to describe the various aspects of using SPs in a pretest/post-test design.

METHODS

We used incognito SPs as known patients in a controlled trial to assess the practice behaviour of 49 trainees. The SPs received a 2-day training in role playing and completing checklists. We compiled comprehensive practice information folders of each practice to be visited. Real personal data and faked medical data of SPs were inserted in the filing system of each practice to be visited. Apart from SP roles with slightly different reasons-for-encounter and different SPs, the same training protocol, checklists and practice information folders were used in the post-test.

RESULTS

The SPs carried out 287 visits in 50 practices. All practices prepared the patient records for the SP visits in a fairly authentic practice style. Trainees detected the SP in 74 visits. The main reasons for detection were imperfections in patient records and aspects of SP roles or role playing.

CONCLUSION

Fielding incognito SP visits as known patients was feasible but labour-intensive. Preparing the SP patient records and familiarising SPs with the interior of practices represented new elements in the use of SPs. The pretest/post-test format made their use more efficient instead of complicating it.

Integrating clinical practice guidelines into daily practice: impact of an interactive workshop on drafting of a written action plan for asthma patients

INTRODUCTION

Written action plans (WAPs) are instructions that enable asthmatics to manage their condition appropriately and are recommended by current asthma clinical practice guidelines (CPGs). However, general practitioners (GPs) rarely draft WAPs for their patients. An interactive, case-based workshop for asthma, combined with an objective structured clinical examination (OSCE), was developed to increase GPs' knowledge and use of WAPs in Québec.

METHODS

The study compared 24 GPs receiving an OSCE preworkshop and 12 months post-workshop (group 1) with 16 GPs receiving an OSCE preworkshop and at 6 and 12 months post-workshop (group 2). Participants received no feedback on their OSCE performance. During the workshop, which presented a preformatted tool to aid drafting of the WAPs, all 40 GPs worked individually and in small groups to answer questions on a hypothetical clinical case and then discussed the responses with a facilitator and an asthma specialist.

RESULTS

Only group 2 GPs showed a significant improvement in knowledge of WAPs 12 months post-workshop (p = .01). The likelihood of prescribing WAPs to patients increased in group 2 to a degree approaching statistical significance (p = .070), and there was a borderline nonsignificant trend for prescribing practice to improve more among group 2 GPs than among group 1 GPs (p = .052). There was also a nonsignificant trend for 6-month OSCE to increase attendance at the 12-month OSCE.

DISCUSSION

An interactive workshop employing a preformatted WAP tool and a reinforcing OSCE at 6 months yielded improved knowledge of WAPs at 1 year. Although originally developed as a form of evaluation, the OSCE appears to have formative value even when correction is not provided and may increase the effectiveness of continuing medical education interventions to enhance CPG implementation.

Evaluation of an osteoporosis self-referral program to enhance management outcomes

Osteoporosis is a common but undertreated condition. While bone density is known to predict fracture risk, there is currently no economical way of measuring general population risk. This study examined whether an economical screening technique could improve diagnosis and lead to appropriate outcomes in the management of osteoporosis. A self-referral program was established to provide women with information about osteoporosis and an individualized risk assignment. A high-risk designation was provided for postmenopausal women below the lower tertile of a calcaneal broadband ultrasound attenuation (BUA) (< or = 58 db/MHz) or those with a SCORE value > or = 6. A telephone survey was conducted 6 mo after program registration to evaluate osteoporosis management outcomes. Of 271 women surveyed, 181 (67%) were postmenopausal and thus potential candidates for estrogen replacement, and 21% and 63% were assigned a high-risk profile by either calcaneal ultrasound or SCORE, respectively. Women at higher risk were significantly more likely to discuss osteoporosis with their family physician (p < 0.001), and to undergo further testing with dual X-ray absorptiometry measurement (p < 0.005). Utilization of estrogen replacement by postmenopausal women at time of survey was significantly increased compared to registration (p = 0.01). The self-referral program positively impacted decisions women made about preventing osteoporosis, particularly if they considered themselves at higher risk.

Strategies for the prevention of hip fracture

Hip fractures are associated with 10% to 20% excess mortality in the first year and cause functional disability in most survivors. An estimated 17% of white women in the United States will sustain a hip fracture after the age of 50 years. Despite the availability of evidence-based guidelines for hip fracture prevention, routine screening and preventive measures have not been incorporated into standard primary care practice. Many physicians lack adequate knowledge to initiate bone mineral density testing and treatment with preventive medications to decrease the incidence of osteoporosis and fractures. Furthermore, patients are less likely to request information about bone health than about diseases for which systematic screening and prevention protocols have been established. This review describes preventive measures to decrease hip fracture in postmenopausal women, including screening by bone mineral density testing, risk factor assessment, and chemoprevention. Existing guidelines are summarized, and dilemmas regarding their implementation are discussed.

Establishing priorities for hospital education

INTRODUCTION

Integrating the emerging evidence base for medicine, measuring quality of care, reducing medical errors, and understanding the managed care environment all present new challenges to the processes and outcomes of hospital education. Establishing priorities for hospital-based education is critical as cost containment measures influence education budgets. This article examines the learning modalities preferred by health care educators in their own educational experiences and the implications of preferences for newly emerging roles in hospital education.

METHODS

To determine if a typology based on learning modalities has utility for helping health care educators establish priorities, a pilot study was conducted using three groups of health care educators. Subjects in the pilot study completed a questionnaire that asked them to list three examples of their most important recent learning experiences.

RESULTS

Sixty-six subjects reported 181 learning experiences. These 181 learning experiences were divided into four learning modalities displayed in a 2 x 2 matrix based on "motivation" and "management" of learning as follows: 97 in problem solving, 52 in training, 16 in schooling, and 16 in hobbying.

DISCUSSION

From this study, it appears that problem solving is the prime learning modality of adults in health care, and departments of hospital education would be well advised to focus on practical learning and problem solving when developing their educational activities. This finding supports the use of selected modalities of education in meeting the new challenges of hospital education.

Does patient partnership in continuing medical education (CME) improve the outcome in osteoporosis management?

INTRODUCTION

There is an apparent gap between physicians' knowledge and their practical application of such knowledge. Educating patients to educate physicians toward improved care has been shown to be effective in selected settings. This study describes the influence of an active community education program on changing physician behavior.

METHODS

A total of 672 schoolteachers were interviewed and screened for detection of osteoporosis risk factors. The teachers were educated about osteoporosis and the significance of bone mass densitometry (BMD) in its management and returned to their physicians with the results of their screening. One group of their physicians received didactic lectures on the results of the screening and the impact of appropriate management on the outcome of osteoporosis. The second group of physicians did not. Six months after the initial interview and screening, patients received a mailed survey.

RESULTS

After 6 months, 258 of the teachers who had visited their physicians were resurveyed. The percentage of BMD tests ordered by physicians rose from 8.6% to 33% in both the intervention and control physician groups. Findings at the 6-month survey indicated that physicians initiated the osteoporosis discussion an average of 18% of the visits and patients did an average of 60%, prompting the physician to order a BMD test.

DISCUSSION

In this study, there was no evidence that didactic lectures affect physician behavior in ordering BMD. There were, however, compelling indications that patient education, which included written screening results, enhanced physician-patient dialogue, resulting in more BMD orders.