An objective structured clinical examination to assess problem-based learning

What's been trending with OSCEs in pharmacy education over the last 20 years? A bibliometric review and content analysis

BACKGROUND

Objective structured clinical examinations (OSCEs) remain an integral part of pharmacy education. This study aimed to characterize key researchers, areas, and themes in pharmacy education OSCEs using a bibliometric review with content analysis.

METHODS

A bibliometric review was conducted on literature from over 23 years from January 2000 to May 2023. Articles focusing on any type of OSCE research in pharmacy education in both undergraduate and postgraduate sectors were included. Articles were excluded if they were not original articles or not published in English. A summative content analysis was also conducted to identify key topics.

RESULTS

A total of 192 articles were included in the analysis. There were 242 institutions that contributed to the OSCE literature in pharmacy education, with the leading country being Canada. Most OSCE research came from developed countries and were descriptive studies based on single institution data. The top themes emerging from content analysis were student perceptions on OSCE station styles (n = 98), staff perception (n = 19), grade assessment of OSCEs (n = 145), interprofessional education (n = 11), standardized patients (n = 12), and rubric development and standard setting (n = 8).

IMPLICATIONS

There has been a growth in virtual OSCEs, interprofessional OSCEs, and artificial intelligence OSCEs. Communication rubrics and minimizing assessor variability are still trending research areas. There is scope to conduct more research on evaluating specific types of OSCEs, when best to hold an OSCE, and comparing OSCEs to other assessments.

Student Performance on an Objective Structured Clinical Exam Delivered Both Virtually and In-Person

OBJECTIVE

Passing a milestone objective structured clinical examination (OSCE) is a graduation requirement for the University of Waterloo Pharmacy students. In January 2021, the milestone OSCE was offered concurrently both virtually and in-person, with students being able to choose their desired format. The purpose of this study was to compare student performance between the 2 formats and to identify factors that may have predicted student choice of format.

METHODS

Objective structured clinical examination scores for in-person and virtual exam-takers were compared using 2-tailed independent t tests with Bonferroni correction. Pass rates were compared using χ2 analysis. Prior academic performance variables were analyzed to identify predictors of the chosen exam format. Student and exam personnel surveys were used to capture OSCE feedback.

RESULTS

A total of 67 students (56%) participated in the in-person OSCE, and 52 students (44%) participated virtually. There were no significant differences in overall exam averages or pass rates between the 2 groups. However, virtual exam-takers scored lower in 2 of 7 cases. Previous academic performance did not predict the choice of exam format. Feedback surveys indicated that the exam organization was perceived as a strength regardless of format, but in-person students felt more prepared for the exam than virtual exam-takers with technical challenges and difficulty navigating station resources being noted as barriers in the virtual offering.

CONCLUSION

Virtual and in-person administration of a milestone OSCE resulted in similar student performance, with slightly lower performance on 2 individual case scores with virtual delivery. These results may inform the future development of virtual OSCEs.

Shifting to Authentic Assessments? A Systematic Review of Student Perceptions of High-Fidelity Assessments in Pharmacy

OBJECTIVE

Despite becoming increasingly popular, there is no comprehensive review on high-fidelity assessments in pharmacy education that has a focus on the perceptions and experiences of students. This systematic review investigates the student acceptability of high-fidelity simulation for summative assessments in pharmacy education and provides recommendations regarding high-fidelity simulation practices.

FINDINGS

The search yielded 37 studies in total. The articles fell into 3 distinct categories: objective structured clinical examinations (N = 25); face-to-face simulation assessments (N = 9), and augmented reality assessments (N = 3). Most high-fidelity assessments were well received with most students agreeing they had a vital role in assessing the application of clinical knowledge, even though they are stressful. Students prefer high-fidelity assessments to be face-to-face rather than online and also prefer when simulated patients whom they are unfamiliar with are used. Students also expressed a need to be well-prepared for the assessment regarding the logistics of the exam and technology use.

SUMMARY

High-fidelity simulation is likely to be increasingly important in the assessment of the knowledge and skills of pharmacy students and student perception is an important factor to consider when developing such assessments. Reducing stress associated with high-fidelity assessments could include familiarizing students with task logistics or technology used prior to the assessment, using external simulated patients, and having face-to-face assessments and practice sessions.

Outcomes of inquiry-based learning in health professions education: a scoping review

Background

Open inquiry-based learning (IBL) that aims to foster higher-level thinking, is defined by students formulating their own questions and learning through exploration. The present study aimed to summarize the breadth of metrics used to evaluate health professions trainees in open IBL curricula.

Methods

We conducted a scoping review to identify publications detailing trainee outcomes in open IBL initiatives in health professions education. We queried five databases and included studies which described interventions with five phases of IBL (orientation, conceptualization, investigation, conclusion, and discussion). We completed abstract and full text reviews in duplicate. Data were collated and summarized.

Results

From 3030 record, 21 studies were included in the final extraction (k = 0.94), with nine involving physician trainees and twelve involving nursing trainees. Three studies used validated data collection tools to measure student inquiry behavior, and a single study used a validated data collection tool to measure critical thinking abilities. Most studies (n = 11) reported trainee self-reported satisfaction or perceived gain of skills as the primary outcome. All four studies using validated tools reported high scores in inquiry behaviors at the end of the curriculum and results on critical thinking skills were mixed. One study collected serial data, while remaining studies collected pre-post or post-only data.

Conclusion

IBL has the potential to cultivate a climate of curiosity among health professions learners. However, studies have relied heavily on subjective outcomes. Limited studies reported standardized measures of inquiry behaviors suggest favorable results. Curriculum innovations using IBL could make use of existing tools to better understand their impact on students' inquiry-oriented skills.

Evidence for commonly used teaching, learning and assessment methods in contact lens clinical skills education

INTRODUCTION

Evidence based practice is now an important part of healthcare education. The aim of this narrative literature review was to determine what evidence exists on the efficacy of commonly used teaching and learning and assessment methods in the realm of contact lens skills education (CLE) in order to provide insights into best practice. A summary of the global regulation and provision of postgraduate learning and continuing professional development in CLE is included.

METHOD

An expert panel of educators was recruited and completed a literature review of current evidence of teaching and learning and assessment methods in healthcare training, with an emphasis on health care, general optometry and CLE.

RESULTS

No direct evidence of benefit of teaching and learning and assessment methods in CLE were found. There was evidence for the benefit of some teaching and learning and assessment methods in other disciplines that could be transferable to CLE and could help students meet the intended learning outcomes. There was evidence that the following teaching and learning methods helped health-care and general optometry students meet the intended learning outcomes; clinical teaching and learning, flipped classrooms, clinical skills videos and clerkships. For assessment these methods were; essays, case presentations, objective structured clinical examinations, self-assessment and formative assessment. There was no evidence that the following teaching and learning methods helped health-care and general optometry students meet the intended learning outcomes; journal clubs and case discussions. Nor was any evidence found for the following assessment methods; multiple-choice questions, oral examinations, objective structured practical examinations, holistic assessment, and summative assessment.

CONCLUSION

Investigation into the efficacy of common teaching and learning and assessment methods in CLE are required and would be beneficial for the entire community of contact lens educators, and other disciplines that wish to adapt this approach of evidence-based teaching.

Comparison of student performance in therapeutics and communications courses to outcomes of objective structured clinical examinations: A retrospective analysis

INTRODUCTION

The objective was to examine the association between course grades and objective structured clinical examination (OSCE) performances in a pharmacy student cohort.

METHODS

A retrospective analysis of demographics, final grades in the Therapeutics I through IV and Communications courses, and multiple OSCE scores (therapeutic knowledge and general communication skills [GCS]) was performed for two cohorts.

RESULTS

Female students scored higher than male students on the warfarin OSCE therapeutic knowledge (P = .008) and GCS scores (P = .02). Age was inversely correlated with warfarin OSCE therapeutic knowledge score (P = .001). The warfarin OSCE therapeutic knowledge score was positively related to Therapeutics II final grades (P < .001). The Communications course final grade was positively correlated to the warfarin OSCE faculty-rated GCS score (P = .005). Therapeutics final grades were not significantly related to station scores of a multi-station OSCE (P > .05). The final regression model included gender and Therapeutics II final grade and explained 6% of the variance in warfarin OSCE therapeutic knowledge scores.

CONCLUSIONS

A number of significant associations were found between demographics, final course grades, and specific OSCE scores. A regression model was significant, but only explained a low percentage of the warfarin OSCE therapeutic knowledge score variance, suggesting other factors not evaluated had a greater effect on scores. This research suggests that OSCEs play an important role in demonstrating student competency in educational domains other than knowledge and brings forth new data suggesting that age and/or gender may influence OSCE performance.

Evolving to Objective Structured Clinical Exams (OSCE): Transitional experience in an undergraduate pharmacy program in Kuwait

BACKGROUND

Objective Structured Clinical Exams (OSCEs) can assess professional competencies in a structured manner and facilitate objective evaluation of clinical performance. With limited data from the Eastern Mediterranean region, this study aims to describe the development, implementation, and evaluation of OSCEs for final year pharmacy students in Kuwait. The study also aims to compare students' performance in two academic years (2015-2016 and 2016-2017).

METHODS

The design, implementation, and evaluation of the competency-based OSCE followed a 3-phase systematic evidence-based approach. The development phase involved establishing an OSCE working group to develop a blueprint and scoring rubrics and to organise assessors and standardised patient/physician training. The implementation phase involved conducting formative and summative OSCEs. The evaluation phase involved undertaking student and staff perception surveys.

RESULTS

The overall students' OSCE scores for the academic years 2015-2016 and 2016-2017 were (median (interquartile range)) (71.6%, 32.2) and (60.0% (30.7)) and respectively (p < 0.0001). The average students' performance score was high in stations covering 'patient consultation and diagnosis' competency (71.4% (95% CI: 66.7-73.3)) and lower in stations covering 'monitoring of medicine therapy' competency (50.0% (95% CI: 33.3-66.7)). Students perceived stations covering 'monitoring medicines therapy' and 'assessment of medicine' as difficult. However, staff perceived stations related to 'patient consultation and diagnosis' competency as the easiest. Students reported that the OSCE was a positive experience as it provided them an opportunity to practice real life scenarios in a safe learning environment.

CONCLUSION

The OSCE helped to identify the level of competency of students prior to graduation and areas to improve in the curriculum.

Interactive Clinical Avatar Use in Pharmacist Preregistration Training: Design and Review

BACKGROUND

Virtual patients are interactive computer-based simulations that are being increasingly used in modern health care education. They have been identified as tools that can provide experiential learning and assessment in a standardized and safe environment. However, the study of high-fidelity virtual patients such as interactive clinical avatars within pharmacy is limited.

OBJECTIVE

The aim of this paper is to describe the design and review of three interactive clinical avatar simulations as part of pharmacist preregistration training.

METHODS

A multistep design approach was taken to create interactive clinical avatar simulations on the topics of emergency hormonal contraception (EHC), calculation of renal function, and childhood illnesses. All case studies were reviewed by registered pharmacists to establish content and face validity. The EHC case study and data collection questionnaire were also reviewed by a purposive sample of preregistration trainees and newly qualified pharmacists. The questionnaire used Likert ranking statements and open-ended questions to obtain users' feedback on the design, usability, and usefulness of the interactive clinical avatars as learning tools. Descriptive statistics and content analysis were undertaken on the data.

RESULTS

Ten preregistration trainees and newly qualified pharmacists reviewed the EHC interactive clinical avatars and data collection questionnaire. The data collection questionnaire was associated with a Cronbach alpha=.95, demonstrating good reliability. All three interactive clinical avatar simulations were reported as usable and appropriately designed for preregistration training. Users perceived they were developing skills and knowledge from the simulations. The high-fidelity nature of the avatars and relevance of the simulations to real-life practice were reported as aspects that encouraged the application of theory to practice. Improvements were suggested to ensure the simulations were more user-friendly.

CONCLUSIONS

The design and creation of the interactive clinical avatar simulations was successful. The multistep review process ensured validity and reliability of the simulations and data collection questionnaire. The in-depth explanation of the design process and provision of a questionnaire may help widen the use and evaluation of interactive clinical avatars or other simulation tools in pharmacy education. The interactive clinical avatars were reported as novel learning tools that promoted experiential learning and allowed users to feel like they were engaging in real-life scenarios, thus developing transferable knowledge and skills. This may be potentially beneficial for many health care training courses as a way to provide standardized experiences promoting active learning and reflection.

Virtual patients as a tool for training pre-registration pharmacists and increasing their preparedness to practice: A qualitative study

Virtual patients are an active learning pedagogical tool which simulate clinical scenarios in a three-dimensional environment. Their use in pharmacy education is under-researched in comparison to other healthcare professions. In the United Kingdom, pre-registration training refers to a year of workplace based training which pharmacy graduates must complete prior to professional registration as pharmacists. This study aimed to evaluate pre-registration pharmacists' perceptions on the integration, usefulness and enjoyment of completing virtual patient simulations or non-interactive case studies as part of their training. Pre-registration trainees completed three virtual patient simulations or three non-interactive case studies on the topics of: emergency hormonal contraception, renal function and childhood illnesses. Telephone interviews were conducted with twenty pre-registration pharmacists, exploring their perspectives on the use of the virtual patient or non-interactive case studies. Data was analysed using the five-stage framework approach. Four main themes emerged from the data: case study design; usefulness of the case studies as a training tool; support in pre-registration training; utility of the learning tools. Trainees also identified technical issues they had experienced while completing the virtual patient simulations, specifically with keyword recognition. Pre-registration trainees who used the virtual patients provided comments relating to the novelty, realism and enjoyment in completing them. Trainees in both groups reported developing knowledge and skills from completing the case studies; those who used the virtual patient commented on the development of communication skills and an increase in confidence for practice and those who used the non-interactive cases focused on knowledge acquisition and numeracy. Participants were enthusiastic about virtual patients as a novel training tool which provided an opportunity for learners to practice realistic scenarios in a safe environment. Virtual patients offer the potential to 'bridge the gap' in pharmacist pre-registration sector-related training variation, promote learning through reflection on doing and increase overall preparedness for practice.

Actual vs. Perceived Competency Development-How Can Virtual Patients Impact Pharmacist Pre-Registration Training?

Virtual patients are an active learning pedagogical tool that simulate clinical scenarios. There is an established disparity in pre-registration examination pass rates depending on whether individuals complete their training in a community or hospital pharmacy. This study aimed to evaluate virtual patient (VP) and non-interactive (NI) case studies, concerning knowledge, skill and confidence development of pre-registration pharmacist trainees. A quasi-experimental evaluation was conducted. Pre-registration pharmacists completed three VP or NI case studies. Each case study was associated with a pre-and post-knowledge quiz. Pre-registration trainees were invited to complete a questionnaire consisting of Likert ranking statements and open-ended questions on the case study features, usability and individual development. Both learning tools significantly improved trainees' knowledge on the topic areas (except for the NI group in case study 3). Although no significant differences in knowledge improvement were identified between the learning tools, trainees who used the VP reported the development of a wider knowledge base and skill set, an increase in confidence for practice and an opportunity to apply their learning. The sector in which pre-registration pharmacists were completing their training (community or hospital) had a significant impact on knowledge improvement in the three case studies. Future research evaluating VPs with pre-registration and qualified pharmacists should be conducted to explore their benefits and establish their effectiveness as learning tools.

Reliability and Validity of a Checklist to Evaluate Student Performance in a Problem-Based Learning Group

Objective. To validate a problem-based learning (PBL) evaluation checklist to assess individual Doctor of Pharmacy (PharmD) students' performance in a group. Methods. In 2013, a performance checklist was developed and standardized. To evaluate the reliability and discriminant validity of the checklist, pharmacy students' evaluation scores from 2015-2016 were assessed along with overall program grade point averages (GPA), and scores on knowledge and problem-solving examinations. Predictive analysis software was used to analyze the data. Results. Seventy facilitators generated 1506 evaluation reports for 191 (90 third-year and 101 second-year) students over eight PBL cases. The mean (SD) total score was 40.6 (2.5) for P3s and 39.1 (2.7) for P2s out of a possible 44.2 points. Students' scores improved each semester. Interrater reliability based on intraclass correlation coefficient for all cases was 0.67. Internal reliability as determined by Cronbach alpha was >0.7 for all binary checklist items across all cases. Discriminant validity assessed using Pearson correlation coefficient showed that the total score from the checklist did not correlate with knowledge or problem-solving examination scores. Conclusion. This unique PBL checklist proved to be a reliable and valid tool to assess student performance in small group sessions in a PharmD curriculum.

Evaluation of a Hospital Pharmacist Annual Competency Program

Purpose: The purpose of this project was to evaluate a pharmacist annual competency evaluation (PACE) program for pharmacists at a single site. The results of this evaluation will be utilized to understand the effectiveness of PACE and provide suggestions for programmatic improvement. Methods: The primary outcome of this study was to evaluate the change in pharmacist composite self-efficacy (PSE) scores before and 1 month after the PACE program. The composite score was composed of self-efficacy related to 5 different clinical tasks. The 5 tasks selected were advanced cardiac life support, chemotherapy, dofetilide, patient-controlled analgesia pump, and alteplase (tPA). Secondary outcomes included evaluating each component of the composite score at 1 month, changes in composite score and components at 1 year, and changes in the generalized self-efficacy (GSE) survey at 1 month. Last, themes were identified from a feedback questionnaire after completion of PACE. Results: For the primary endpoint, PSE composite scores increased by 12.6% from pre-PACE to post-PACE at 1 month (79.6 ± 12.2 vs 89.7 ± 5.8; P < .001). All components of the composite PSE score showed a statistically significant improvement from baseline to 1 month. At 1 year, tPA was the only clinical task that was statistically different from baseline. GSE scores increased by 11.4% from pre-PACE to post-PACE (31.6 ± 3.6 vs 35.4 ± 2.8). No pharmacists required remediation. Conclusion: The competency assessment program led to improvements in pharmacist self-efficacy in dealing with low-volume, high-risk clinical pharmacy tasks. Future studies could assess various forms of competency assessment and timing of these programs to determine the most effective way to ensure pharmacist competency.

Implementation of objective structured clinical examinations (OSCEs) in a doctor of pharmacy program

BACKGROUND AND PURPOSE

Objective structured clinical examinations (OSCEs) are an assessment solution that can be adapted in multiple ways within the doctor of pharmacy (PharmD) curriculum to assess clinical and communication skills and to improve teaching and learning across the curriculum.

EDUCATIONAL ACTIVITY AND SETTING

The University of Florida College of Pharmacy formed an OSCE Assessment Committee, wrote blueprints for the OSCEs aligned with Entrustable Professional Activities (EPAs), and delivered OSCEs in the first and third professional years of the PharmD program.

FINDINGS

Overall, students performed well on both the first year OSCE and third year OSCE (96 to 97% pass rate on first attempt).

SUMMARY

OSCEs provide a number of benefits to students, faculty, and preceptors. Students are afforded the opportunity to demonstrate the knowledge and skills they have acquired during pharmacy school. Likewise, faculty are able to determine if the skills they are teaching are being retained by students. Faculty are also able to assess if students are ready for pharmacy practice rotations. OSCEs also identify areas within the curriculum which may need reinforcement or review. In order for OSCEs to provide benefit to students and to the college, thought should be given as to what skills OSCEs will assess, how they will be developed and administered, and how they will be graded and used to determine progression through the curriculum.

Current Trends and Opportunities for Competency Assessment in Pharmacy Education-A Literature Review

An increasing emphasis on health professional competency in recent times has been matched by an increased prevalence of competency-based education models. Assessments can generate information on competence, and authentic, practice-based assessment methods are critical. Assessment reform has emerged as an academic response to the demands of the pharmacy profession and the need to equip graduates with the necessary knowledge, skills and attributes to face the challenges of the modern workforce. The objective of this review was to identify and appraise the range of assessment methods used in entry-level pharmacy education and examine current trends in health professional assessment. The initial search located 2854 articles. After screening, 36 sources were included in the review, 13 primary research studies, 12 non-experimental pharmacy research papers, and 11 standards and guidelines from the grey literature. Primary research studies were critically appraised using the Medical Education Research Study Quality Instrument (MERSQI). This review identified three areas in pharmacy practice assessment which provide opportunities for expansion and improvement of assessment approaches: (1) integrated approaches to performance assessment; (2) simulation-based assessment approaches, and; (3) collection of validity evidence to support assessment decisions. Competency-based assessment shows great potential for expanded use in pharmacy, but there is a need for further research and development to ensure its appropriate and effective use.

The Need to Strengthen the Role of the Pharmacist in Sri Lanka: Perspectives

The role of the pharmacist in healthcare has evolved greatly over the last half-century, from dispensing to providing direct patient-oriented activities not associated with dispensing. However, pharmacist-led healthcare services in Sri Lanka must undergo reform to fully take advantage of their expertise and training in medicine management and related outcomes in Sri Lankan patients. As befits a profession's role development and value, professional and educational standards for pharmacists need ongoing development and growth. Currently, university curricula and continuing professional education in Sri Lanka require further development and optimisation to provide the theoretical and practical knowledge and skills regarding quality use of medicines and patient-oriented care. Furthermore, pharmacists' roles in Sri Lankan hospital and community pharmacist settings need to be recognised and should include the pharmacist as an integral part of the multidisciplinary healthcare team in Sri Lanka. Studies from developed countries and some developing countries have demonstrated that expanded pharmacists' roles have had a significant positive cost-effective impact on the population's health. Therefore, the availability of qualified Sri Lankan pharmacists trained to deliver expanded professional services accompanied by greater pharmacist integration into healthcare delivery is crucially important to ensure quality use of medicines within the Sri Lankan healthcare system and optimise the medication-related needs of Sri Lankans.

Advancing Critical Thinking Through Learning Issues in Problem-Based Learning

Health professions educators are increasingly urged to use learning designs that promote critical thinking and the development of interpersonal competencies. Problem-based learning (PBL) has a long, albeit contested, history as a collaborative and deep think-aloud process that participants use to reach conclusions about medical cases. In order to make progress, participants must assess what they do not know and what they must learn in order to continue. Answering these learning issues (LI) requires self-direction and cognitive presence. This study analyzes the discussions that participants used in the reporting phase of the LI process in an 8-week PBL module on cardiac-renal systems. Data were drawn from 10 class sessions and analyzed for critical thinking using a model based on Garrison and Newman et al. Participants at first presented LI reports didactically but over time initiated active learning strategies. The findings indicate large increases in the numbers of LI reports in which participants engaged in collaborative thinking. There were also large increases in the amount of time devoted to critical thinking as participants aligned the LI process more closely with the intent of PBL. Participants' identity development as experts also underwent changes and the fluidity of the expert roles increased. Thoughtful design of the LI process can help learners develop the habitus of self-direction and collaborative critical thinking that they need in order to develop clinical reasoning.

PharmAdhere: training German community pharmacists with Objective Structured Clinical Examinations

Background Pharmacists who engage in Pharmaceutical Care need skills to optimise responsible medication use and increase medication adherence. Objectives We developed and evaluated a blended-learning programme for German community pharmacists that focused on conducting consultations in chronic diseases. Setting Community pharmacists in Germany. Method Interventional study with pre-post design. We combined e-learning with Objective Standardised Clinical Examinations (OSCEs) for emergency situations, initiation/implementation of medication therapy and detection of symptoms of four chronic diseases. Specific procedures were defined in the Pharmaceutical Action Plan. Skills were measured with a global analytical marking sheet derived from the Medication Related Consultation Framework and scored with the Canadian criticality/relevancy matrix. Time limits matched real practice scenarios. Main outcome measures Changes in knowledge (difference of test results before and after e-learning) and changes in skills (difference in scores of the OSCEs before and after training). Results 22 out of the 26 pharmacists enrolled, completed the study. The number of correctly answered questions increased significantly after the e-learning for all four indications with a mean number of additional correct answers between 3.86 and 4.9 points out of 15 (p < 0.001). The sums of the analytical checklist points in percentages increased significantly in all topics from the baseline summative OSCE to the final summative OSCE between 6.14 and 31.85% (p < 0.001). The maximum duration of consultation per patient was less than 10 min in all OSCEs. Conclusion The use of e-learning and OSCEs was well received by participants and is a successful method to deliver practical Pharmaceutical Care training.

Peer-assisted Learning: Clinical Skills Training for Pharmacy Students

Objective. To assess the impact of peer-teaching on student scores and confidence when preparing for a final objective structured clinical examination (OSCE) within a Doctor of Pharmacy program. Methods. First-year pharmacy students (n=45) attended a peer-led training session led by upperclassmen (n=17) on a variety of clinical skills to be assessed on a final course OSCE. Their scores were collected and compared to students who did not attend the training. Confidence scores were also evaluated using voluntary pre- and post-surveys. Results. An overall 3% increase in scores was recorded from the objective skills examination. Student confidence scores also increased for each of the skills evaluated with an overall improvement of 1.1 on a 5-point Likert scale. Conclusion. Peer-assisted learning was effective in increasing student performance and confidence in the OSCE. Based on the positive results, the peer-led training event will be improved upon and used again in the future.

Situational awareness within objective structured clinical examination stations in undergraduate medical training - a literature search

BACKGROUND

Medical students may not be able to identify the essential elements of situational awareness (SA) necessary for clinical reasoning. Recent studies suggest that students have little insight into cognitive processing and SA in clinical scenarios. Objective Structured Clinical Examinations (OSCEs) could be used to assess certain elements of situational awareness. The purpose of this paper is to review the literature with a view to identifying whether levels of SA based on Endsley's model can be assessed utilising OSCEs during undergraduate medical training.

METHODS

A systematic search was performed pertaining to SA and OSCEs, to identify studies published between January 1975 (first paper describing an OSCE) and February 2017, in peer reviewed international journals published in English. PUBMED, EMBASE, PsycINFO Ovid and SCOPUS were searched for papers that described the assessment of SA using OSCEs among undergraduate medical students. Key search terms included "objective structured clinical examination", "objective structured clinical assessment" or "OSCE" and "non-technical skills", "sense-making", "clinical reasoning", "perception", "comprehension", "projection", "situation awareness", "situational awareness" and "situation assessment". Boolean operators (AND, OR) were used as conjunctions to narrow the search strategy, resulting in the limitation of papers relevant to the research interest. Areas of interest were elements of SA that can be assessed by these examinations.

RESULTS

The initial search of the literature retrieved 1127 publications. Upon removal of duplicates and papers relating to nursing, paramedical disciplines, pharmacy and veterinary education by title, abstract or full text, 11 articles were eligible for inclusion as related to the assessment of elements of SA in undergraduate medical students.

DISCUSSION

Review of the literature suggests that whole-task OSCEs enable the evaluation of SA associated with clinical reasoning skills. If they address the levels of SA, these OSCEs can provide supportive feedback and strengthen educational measures associated with higher diagnostic accuracy and reasoning abilities.

CONCLUSION

Based on the findings, the early exposure of medical students to SA is recommended, utilising OSCEs to evaluate and facilitate SA in dynamic environments.

Advances in clinical pharmacy education in Germany: a quasi-experimental single-blinded study to evaluate a patient-centred clinical pharmacy course in psychiatry

BACKGROUND

The pharmacy profession has shifted towards patient-centred care. To meet the new challenges it is necessary to provide students with clinical competencies. A quasi-experimental single-blinded teaching and learning study was carried out using a parallel-group design to evaluate systematically the benefits of clinical teaching in pharmacy education in Germany.

METHODS

A clinical pharmacy course on a psychiatric ward was developed and implemented for small student groups. The learning aims included: the improvement of patient and interdisciplinary communication skills and the identification and management of pharmaceutical care issues. The control group participated only in the preparation lecture, while the intervention group took part in the complete course. The effects were assessed by an objective structured clinical examination (OSCE) and a student satisfaction survey.

RESULTS

The intervention group achieved significantly better overall results on the OSCE assessment (46.20 ± 10.01 vs. 26.58 ± 12.91 of a maximum of 90 points; p < 0.0001).The practical tasks had the greatest effect, as reflected in the outcomes of tasks 1-5 (34.94 ± 9.60 vs. 18.63 ± 10.24 of a maximum of 60 points; p < 0.0001). Students' performance on the theoretical tasks (tasks 6-10) was improved but unsatisfying in both groups considering the maximum score (11.50 ± 4.75 vs. 7.50 ± 4.00 of a maximum of 30 points; p < 0.0001). Of the students, 93% rated the course as practice-orientated, and 90% felt better prepared for patient contact. Many students suggested a permanent implementation and an extension of the course.

CONCLUSIONS

The results suggest that the developed ward-based course provided learning benefits for clinical skills. Students' perception of the course was positive. Implementation into the regular clinical pharmacy curriculum is therefore advisable.

Improved Learning Outcomes After Flipping a Therapeutics Module: Results of a Controlled Trial

PURPOSE

To evaluate the impact on learning outcomes of flipping a pain management module in a doctor of pharmacy curriculum.

METHOD

In a required first-professional-year pharmacology and therapeutics course at the University of Maryland School of Pharmacy, the pain therapeutics content of the pain management module was flipped. This redesign transformed the module from a largely lecture-based, instructor-centered model to a learner-centered model that included a variety of preclass activities and in-class active learning exercises. In spring 2015, the module was taught using the traditional model; in spring 2016, it was taught using the flipped model. The same end-of-module objective structured clinical exam (OSCE) and multiple-choice exam were administered in 2015 to the traditional cohort (TC; n = 156) and in 2016 to the flipped cohort (FC; n = 162). Cohort performance was compared.

RESULTS

Learning outcomes improved significantly in the FC: The mean OSCE score improved by 12.33/100 points (P < .0001; 95% CI 10.28-14.38; effect size 1.33), and performance on the multiple-choice exam's therapeutics content improved by 5.07 percentage points (P < .0001; 95% CI 2.56-7.59; effect size 0.45). Student performance on exam items assessing higher cognitive levels significantly improved under the flipped model. Grade distribution on both exams shifted, with significantly more FC students earning an A or B and significantly fewer earning a D or F compared with TC students.

CONCLUSIONS

Student performance on knowledge- and skill-based assessments improved significantly after flipping the therapeutics content of a pain management module.

Implementation of a skills practical to first-year pharmacy students

BACKGROUND AND PURPOSE

Objective structured clinical examinations (OSCEs) are being implemented in pharmacy schools across the country. As part of a new curriculum, a practical with an OSCE component was implemented for the first time at the West Virginia University School of Pharmacy.

EDUCATIONAL ACTIVITY AND SETTING

A practical with an OSCE component was developed for first-year professional pharmacy students in a foundational pharmacy skills course. The OSCE component assessed insulin and inhaler counseling and blood pressure skills. A written exam covered material from throughout the semester.

FINDINGS

A total of 76 students completed the practical. The average grade was a 27.8 out of 30 points. A total of 62 faculty hours were needed for the implementation of this OSCE.

DISCUSSION AND SUMMARY

An OSCE that assessed a variety of foundational pharmacy skills was designed and implemented to first year pharmacy students using limited faculty time and resources.

Incorporating age-related special populations into a pharmacy skills laboratory course sequence

BACKGROUND AND PURPOSE

The purpose was this study was to evaluate the incorporation of geriatric and pediatric therapeutic concepts into a pharmacy skills laboratory (PSL) course sequence. Secondary outcomes were to describe the incorporation and evaluate the assessment effectiveness of these topics. Educational activity and setting Teaching materials and assessment data from the PSL course were evaluated for inclusion of content regarding age-related special populations.

FINDINGS

Materials reviewed from 205 PSL sessions held over four years demonstrated 89 (41.5%) sessions incorporated age-related topics using a variety of methods. Quiz and exam item performance was acceptable with an average difficulty index of 77.2% ± 21.3 and 85.8% ± 12.8, respectively. The average objective structured clinical exam (OSCE) score was also satisfactory at 83.2% ± 3.

DISCUSSION AND CONCLUSIONS

Age-related topics were included in several PSL sessions. These incorporation strategies were deemed to be effective as students performed well on written assessments and OSCEs. Additional PSL sessions were identified as potential targets for incorporating age-related special populations into future PSL offerings.

How do undergraduate STEM mentors reflect upon their mentoring experiences in an outreach program engaging K-8 youth?

BACKGROUND

Many university students are becoming involved in mentoring programs, yet few studies describe the impact of mentoring on the mentor. Additionally, many studies report that students graduating from college are not prepared to enter the workforce in terms of key career skills and/or content knowledge. Herein, we examine the impact of our program, NE STEM 4U (Nebraska Science, Technology, Engineering and Math for You), in which undergraduate (UG) mentors engage K-8 youth in after-school STEM experiments. The UGs reflected upon their experiences using post-mentoring evaluations, 12- and 24-week interviews, and exit surveys. Many of the questions asked of the mentors related directly to their own professional development, such as self-evaluation of communication, organization, and problem-solving skills, while other questions related to content knowledge and reflection.

RESULTS

Post-mentoring, UGs reflected on the delivery/teaching significantly more (p ≤ 0.001 for each) than other variables (i.e., their own content knowledge gains, the students' content knowledge gains, scaffolding the lessons, or overall professional growth). By analyzing the evaluations and interviews together, some significant, self-reported gains emerged. For example, 94.15% of the UG reported that the experience was beneficial to their education. Additionally, UG mentors self-reported significant gains (p ≤ 0.01 for each) moving from 12- to 24-weeks in the program in the categories of organization, STEM content knowledge, preparedness to teach, and engagement in the program. However, UG did not report significant gains in dependability. Importantly, when mentors ranked themselves at 24-weeks, they were blinded to (unaware of) the ranking they gave themselves at 12-weeks.

CONCLUSIONS

This study helps to fill a gap in the literature by providing insight into the gains UG mentors report attaining after mentoring to K-8 students. These data suggest that participation by UGs in this program promoted self-reflection as well as self-reported gains related to career preparedness and STEM content knowledge.

Objective structured clinical examination (OSCE) in pharmacy education - a trend

Pharmacy education has undergone a radical change as it evolves towards becoming a more patient oriented profession. With a greater emphasis on problem based teaching and competency, the Objective Structured Clinical Examination (OSCE), supported by its reliability and validity became the gold standard for the evaluation of clinical skills of undergraduate students of medicine and pharmacy worldwide. Core competency evaluation has become a mandatory and critical norm for accountability of educational objectives as the traditional testing tools cannot evaluate clinical competence. Interpersonal and communication skills, professional judgment, skills of resolution etc., may be best assessed through a well- structured OSCE in comparison to oral examinations, multiple choice tests and other methods of assessment. Though OSCEs as an objective method of evaluation offer several advantages to both students and teachers, it also has disadvantages and pitfalls in implementation. This article reviews the OSCE as a trend in pharmacy education.

Using concept mapping to evaluate knowledge structure in problem-based learning

BACKGROUND

Many educational programs incorporate problem-based learning (PBL) to promote students' learning; however, the knowledge structure developed in PBL remains unclear. The aim of this study was to use concept mapping to generate an understanding of the use of PBL in the development of knowledge structures.

METHODS

Using a quasi-experimental study design, we employed concept mapping to illustrate the effects of PBL by examining the patterns of concepts and differences in the knowledge structures of students taught with and without a PBL approach. Fifty-two occupational therapy undergraduates were involved in the study and were randomly divided into PBL and control groups. The PBL group was given two case scenarios for small group discussion, while the control group continued with ordinary teaching and learning. Students were asked to make concept maps after being taught about knowledge structure. A descriptive analysis of the morphology of concept maps was conducted in order to compare the integration of the students' knowledge structures, and statistical analyses were done to understand the differences between groups.

RESULTS

Three categories of concept maps were identified as follows: isolated, departmental, and integrated. The students in the control group constructed more isolated maps, while the students in the PBL group tended toward integrated mapping. Concept Relationships, Hierarchy Levels, and Cross Linkages in the concept maps were significantly greater in the PBL group; however, examples of concept maps did not differ significantly between the two groups.

CONCLUSIONS

The data indicated that PBL had a strong effect on the acquisition and integration of knowledge. The important properties of PBL, including situational learning, problem spaces, and small group interactions, can help students to acquire more concepts, achieve an integrated knowledge structure, and enhance clinical reasoning.

Evaluation of Clinical and Communication Skills of Pharmacy Students and Pharmacists with an Objective Structured Clinical Examination

Objective. To evaluate how effectively pharmacy students and practicing pharmacists communicate and apply knowledge to simulations of commonly encountered patient scenarios using an objective structured clinical examination (OSCE). Design. Second-, third-, and fourth-year pharmacy students completed an OSCE as part of their required courses in 2012 and 2013. All students in both years completed identical OSCE cases. Licensed pharmacists were recruited to complete the OSCE and serve as controls in 2012. A survey assessed student perception and acceptance of the OSCE as well as student confidence in performance. Assessment. Licensed pharmacists had significantly higher clinical and communication skills scores than did pharmacy students. Student progression in communication and clinical skills improved significantly over time. Survey results indicated that students felt the OSCE was well-structured and assessed clinical skills taught in pharmacy school; 86% of students felt confident they could provide these skills. Conclusion. Objective structured clinical examinations can evaluate clinical competence and communication skills among professional students. Implementation of OSCEs may be an effective tool for assessment of the Center for the Advancement of Pharmacy Education domains.

Limited Predictive Utility of Admissions Scores and Objective Structured Clinical Examinations for APPE Performance

OBJECTIVE

To examine the relationship between admissions, objective structured clinical examination (OSCE), and advanced pharmacy practice experience (APPE) scores.

METHODS

Admissions, OSCE, and APPE scores were collected for students who graduated from the doctor of pharmacy (PharmD) program in spring of 2012 and spring of 2013 (n=289). Pearson correlation was used to examine relationships between variables, and independent t test was used to compare mean scores between groups.

RESULTS

All relationships among admissions data (undergraduate grade point average, composite PCAT scores, and interview scores) and OSCE and APPE scores were weak, with the strongest association found between the final OSCE and ambulatory care APPEs. Students with low scores on the final OSCE performed lower than others on the acute care, ambulatory care, and community APPEs.

CONCLUSION

This study highlights the complexities of assessing student development of noncognitive professional skills over the course of a curriculum.

Development of a pharmacy capstone course from focus groups to advanced patient care

OBJECTIVE

To describe the development of a capstone course using qualitative results of focus groups and to determine the impact of the course using a pre- and postcourse surveys.

DESIGN

A course titled Advanced Patient Care was developed using themes emerged from 3 stakeholder focus groups and implemented with case-based sessions, interactive exercises, and Objective Structured Clinical Examinations (OSCEs). Pre- and postcourse surveys were conducted to assess the students' confidence and knowledge in managing 8 commonly-encountered conditions.

ASSESSMENT

During the 2-year course implementation, a total of 169 students participated in the pre- and postcourse surveys (87.6% response rate). The mean total confidence score increased significantly from 54.3 (±9.2) to 69.0 (±8.6, p < 0.001), and the total mean knowledge score increased significantly from 6.3 to 6.9 (p < 0.001).

CONCLUSION

The capstone course, fueled by focus group findings and implemented using interactive sessions and simulations, positively impacted students' confidence and knowledge for clinical practice experiences and professional practice.

A comparison of students' self-assessments with faculty evaluations of their communication skills

OBJECTIVE

To compare students' self-assessment of their communication skills with faculty members' formal evaluation of their skills in a therapeutics course.

METHODS

Over a 3-year period, faculty members evaluated second-year pharmacy students' communication skills as part of a requirement in a therapeutics course. Immediately following an individual oral assessment and again following a group oral assessment, students self-assessed their communication skills using the same rubric the faculty members had used. Students' self-assessments were then compared with faculty members' evaluation of students' communication skills.

RESULTS

Four hundred one (97.3%) students consented to participate in this study. Faculty evaluation scores of students for both the individual and group oral assessments were significantly higher than students' self-assessment scores. Students' self-assessment scores of their communication skills increased from the individual to the group oral assessment.

CONCLUSION

Students' self-assessments of communication skills were consistently lower than faculty members' evaluations. Greater use of oral assessments throughout the pharmacy curriculum may help to improve students' confidence in and self-assessment of their communication skills.

3C3R modified PBL pediatric teaching of Chinese medical students

A Content, Context, Connection and Researching, Reasoning, Reflecting (3C3R) model is a conceptual framework for problem-based learning (PBL) problem design. We introduced the 3C3R-PBL method into a pediatric teaching plan, and evaluated its effectiveness and feasibility. The 3C3R model was applied in a pediatric problem design teaching plan “why the lips turn purple when a baby is crying”. All students were assigned either into a traditional PBL course or into a 9-step 3C3R model PBL course (3C3R-PBL). The performance outcomes of both groups were compared. For the PBL group, the proportion of students scoring ≥4 for content, context, and problem design connection, was 90.8%, 80.3%, and 64.5% respectively, while for tutors, it was 71.4%, 71.4%, and 28.6%; for researching, reasoning, and reflecting, the proportion of students scoring ≥4 was 81.6%, 55.3%, and 40.8%, while for tutors, it was 71.4%, 100%, and 57.1%. The learning difficulty was not considered high with only 31.6% of students and 42.9% of tutors rating the task as difficult. For the 3C3R-PBL group, the proportion of students scoring content, context, and connection, ≥4 was 100%, 98.4%, and 90.5%, while for tutors it was 100%, 100%, 83.3%; for researching, reasoning, and reflecting, the proportion of students scoring ≥4 was 95.2%, 88.9%, and 76.2%, while for tutors it was 100% for all 3 R components. Students and tutors were convinced by the content, case context, research process and reasoning process of both teaching plans, while scores for connection and reflecting were significantly improved when the PBL plan was amended by a 3C3R model (p<0.05) and the case learning difficulty was statistically increased (p<0.05). The 3C3R model, evaluated for the first time in China, was helpful for effective and reliable problem design in a pediatric PBL teaching plan for Chinese students.