Objective structured clinical examination: the assessment of choice

Level of self-efficacy among skill-based allied health students in the University of Santo Tomas-Enriched Virtual Mode of learning: A cross-sectional study

Background

Self-efficacy expresses a learner's perception of how well they can do in the online academic setting. Although students' academic successes can be measured by online learning self-efficacy, there is a scarcity of evidence targeted toward allied health students.

Objective

The primary aim of this study is to determine the level of online self-efficacy among students from skill-based allied health programs. The secondary aim is to correlate online self-efficacy with age, sex, year level, and program of the students.

Methods

This cross-sectional analytic study utilized the Online Learning Self-Efficacy Scale (OLSE) to determine students' level of self-efficacy via a survey conducted with students who underwent the Enriched Virtual Mode of learning. Descriptive statistics was used to characterize the study participants, describe the level of self-efficacy, and compare the subscales of OLSE. Inferential statistics using Spearman's rho was performed to determine the correlation.

Results

A total of 117 respondents with a mean age ± SD of 20.59 ± 0.11 years old, predominantly female (71%) were included in the study. The students from allied health programs had an online self-efficacy overall mean score of 3.83 (SD = 0 . 05 ; range 2.64-5.00) with the use of technology subscales weighted the highest mean score (4.24). Females have a higher average OLSE score (M = 3 . 92 , SD = 0 . 05 ) compared to males. Significant correlation was found between OLSE scores to sex (r = 0 . 260 , p = 0 . 005 ) and year level (r =- 0 . 199 , p = 0 . 031 ) while nonsignificant correlation was found between OLSE scores to the program (r =- 0 . 048 , p = 0 . 604 ) and age (r =- 0 . 123 , p = 0 . 185 ).

Conclusion

Students of allied health programs generally have a very good level of online self-efficacy. This study assists educators in developing proactive strategies and approaches to promote students' self-confidence across all domains and encourage them to adopt a dynamic remote learning-based approach. Educational institutions should use this opportunity to assess how well they have implemented remote learning to ensure educational continuity, especially in times of crisis.

Perceived and actual value of Student-led Objective Structured Clinical Examinations

INTRODUCTION

Student-led Objective Structured Clinical Examinations (OSCEs) provide formative learning opportunities prior to Faculty-led OSCEs. It is important to undertake quality assurance measurements of peer-led assessments because, if they are found to be unreliable and invalid, they may have detrimental impacts. The objectives of this study were to explore and evaluate Student-led OSCEs hosted by fifth-year medical students.

METHODS

Student-led OSCE results were analysed to examine reliability (Cronbach's alpha). The relationship between Student-led and Faculty-led OSCEs was evaluated using linear regression. Qualitative data were acquired by survey and semi-structured interviews and were analysed using an inductive content analysis approach.

RESULTS

In total, 85 (94%) of 91 eligible students consented to study participation. Student-led OSCEs had a low-moderate reliability [Cronbach alpha = 0.47 (primary care) and 0.61 (human reproduction/paediatrics) (HRH)]. A statistically significant, positive relationship between Student-led and Faculty-led OSCE results was observed. Faculty-led OSCE grades increased by 0.49 (95% CI: 0.18, 0.80) to 1.09 (95% CI: 0.67, 1.52), for each percentage increase in Student-led OSCE result. Student-led OSCE participants highly valued the authentic peer-assessed experience. Reported benefits included a reduction of perceived stress and anxiety prior to Faculty-led OSCEs, recognition of learning gaps, contribution to overall clinical competency and facilitation of collaboration between peers.

DISCUSSION

Student-led OSCEs are moderately reliable and can predict Faculty-led OSCE performance. This form of near-peer assessment encourages the metacognitive process of reflective practice and can be effectively implemented to direct further study. Faculties should collaborate with their student bodies to facilitate Student-led OSCEs and offer assistance to improve the quality, and benefits, of these endeavours.

Impact of hospital internships on success in university summative objective structured clinical examinations: Large-scale experience in a French medical school

BACKGROUND

Objective structured clinical examinations (OSCEs) are extensively used in many medical schools worldwide with the stated objective to assess students' clinical skills acquired during internships. The objective of the present study was to assess the factors associated with success in university summative OSCEs, especially the impact of previous hospital internships in corresponding disciplines and supervision during internships.

METHODS

This was a cross-sectional study assessing the results in the summative OSCEs of 4th year medical students during the 2021-2022 academic year in a French medical school. The summative OSCEs included five stations for each student. Each student answered a survey at the end of summative OSCEs about previous internships, the supervision they had and perceived difficulty levels for each station. The scores in each station were assessed according to previous hospital internships in the corresponding discipline. Analysis of predictive factors of success in OSCEs, defined by a score ≥ 10/20 at each station, were performed using univariate and multivariate logistic regression.

RESULTS

Out of the 220 students participating in the summative OSCEs, 182 (83%) answered the survey. One hundred and forty-four (79%) of these students had carried out hospital internships in at least one of the disciplines evaluated during the OSCEs. Students having completed an internship in the corresponding discipline had significantly higher OSCEs scores for interrogation, communication, therapeutic education and procedure stations compared to those who had not. Previous internship in corresponding disciplines was independently associated with success in OSCEs in interrogation (OR 9.45 [1.34-66.8] p = 0.02), clinical examination (OR 6.93 [1.88-25.57] p = 0.004, and therapeutic education (OR 3.09 [1.22-7.82] p = 0.02) stations.

CONCLUSION

Previous hospital internships in the discipline evaluated by the OSCEs are associated with success in summative OSCEs. This reinforces the importance of student involvement during their hospital internships.

Artificial intelligence in academic writing and clinical pharmacy education: consequences and opportunities

The current academic debate on the use of artificial intelligence (AI) in research and teaching has been ongoing since the launch of ChatGPT in November 2022. It mainly focuses on ethical considerations, academic integrity, authorship and the need for new legal frameworks. Time efficiencies may allow for more critical thinking, while ease of pattern recognition across large amounts of data may promote drug discovery, better clinical decision making and guideline development with resultant consequences for patient safety. AI is also prompting a re-evaluation of the nature of learning and the purpose of education worldwide. It challenges traditional pedagogies, forcing a shift from rote learning to more critical, analytical, and creative thinking skills. Despite this opportunity to re-think education concepts for pharmacy curricula several universities around the world have banned its use. This commentary summarizes the existing debate and identifies the consequences and opportunities for clinical pharmacy research and education.

Implementation of standard setting for high-stakes objective structured clinical examinations

BACKGROUND AND PURPOSE

To describe one institution's approach to transformation of high-stakes objective structure clinical examinations (OSCEs) from norm-referenced to criterion-referenced standards setting and to evaluate the impact of these changes on OSCE performance and pass rates.

EDUCATIONAL ACTIVITY AND SETTING

The OSCE writing team at the college selected a modified Angoff method appropriate for high-stakes assessments to replace the two standard deviation method previously used. Each member of the OSCE writing team independently reviewed the analytical checklist and calculated a passing score for active stations on OSCEs. Then the group met to determine a final pass score for each station. The team also determined critical cut points for each station, when indicated. After administration of the OSCEs, scores, pass rates, and need for remediation were compared to the previous norm-referenced method. Descriptive statistics were used to summarize the data.

FINDINGS

OSCE scores remained relatively unchanged when switched to a criterion-referenced method, but the number of remediators increased up to 2.6 fold. In the first year, the average score increased from 86.8% to 91.7% while the remediation rate increased from 2.8% to 7.4%. In the third year, the average increased from 90.9% to 92% while the remediation rate increased from 6% to 15.6%. Likewise, the fourth-year average increased from 84.9% to 87.5% while the remediation rate increased from 4.4% to 9%.

SUMMARY

Transition to a modified Angoff method did not impact average OSCE score but did increase the number of remediations.

Evaluating student performance assessment methods in Objective Structured Clinical Exam: perspectives and comparisons

PURPOSE

This study aims to evaluate how student performance and perspectives changed when the Objective Structured Clinical Exam (OSCE) assessment system was changed from a composite score to discipline-specific grading at the Harvard School of Dental Medicine.

METHODS

The retrospective study population consisted of all students (n = 349) who completed three OSCEs (OSCE 1, 2, and 3) as part of the predoctoral program during the years 2014-2023. Data on the students' OSCE scores were obtained from the Office of Dental Education, and data on students' race/ethnicity and gender were obtained from their admissions data.

RESULTS

The likelihood of a student failing the OSCE after the assessment system change significantly increased with an adjusted odds ratio of 20.12. After the change, the number of failed subjects per student decreased with an adjusted mean ratio of 0.48. Students perceived the OSCE as being less useful after the change. Independent of the grading change, OSCEs 1 and 2 were seen as more useful compared to OSCE 3, which is administered in the last year of the Doctor of Dental Medicine program.

CONCLUSION

The discipline-specific nature of the new assessment system helps focus on specific areas of remediation, rather than blanket remediation used previously, in order to isolate the actual areas of deficiency and to focus remediation efforts so that students can align their learning needs appropriately. Therefore, although the actual number of fails identified increased for the course, the assessment change has allowed for more directed, actionable information to be gained from the OSCE to prepare students to work toward competency standards.

Checklist-based approach to measure milker behavior before and after training

So far, evaluation of training initiatives for dairy farm employees has been limited to the reaction and learning level. The effect of training on dairy farm employees' behavior has not been studied yet. The objective of this study was to determine the effectiveness of online training to change employee behavior toward best-practice routines. An interactive training course related to udder health was developed in 5 modules in Spanish and English: (1) collecting an aseptic milk sample, (2) administering intramammary treatment, (3) deciding on salable milk, (4) treating a cow at dry-off with antibiotics, and (5) administering teat sealants. Participants navigated the modules at their own pace or watched a narrated video. Employees were assessed performing 2 procedures related to dry cow treatment (i.e., treating a cow at dry-off with antibiotics, administering internal teat sealant) by using an objective structured clinical examination. If possible, all employees were scored performing the procedure on 2 cows before and within 2 to 3 wk after the training was applied.

Impact of a Mock OSCE on Student Confidence in Applying the Pharmacists' Patient Care Process

The Medical and Pharmacy Student Collaboration (MAPSC) student organization at the University of Southern California, Alfred E. Mann School of Pharmacy and Pharmaceutical Sciences, created an extracurricular, peer-led, virtual group mock objective structured clinical examination (MOSCE) to expose first-year pharmacy students (P1s) to the Pharmacists' Patient Care Process (PPCP). The purpose of this study is to evaluate the impact of a MAPSC MOSCE on P1s self-reported confidence in applying the PPCP and on patient communication, medication knowledge, and clinical skills. An anonymous, optional, self-reported survey was administered to P1s before and after the event, where they rated their confidence on a scale of 0-100 (0 = not confident, 100 = certainly confident). The statistical analysis was a paired two-tailed t-test with a significance level of p < 0.05. A total of 152 P1s and 30 facilitators attended the MOSCE. One hundred thirty-nine students met the inclusion criteria and were included in the data analysis. There was a statistically significant difference in the change in self-reported confidence for all PPCP components and learning outcomes. The results of our study strongly indicate that introducing P1 students to the PPCP through a MAPSC MOSCE format is a valuable experience.

Undertaking a face-to-face objective structured clinical examination for medical students during the COVID-19 pandemic

INTRODUCTION AND AIMS

Objective structured clinical examinations (OSCEs) play a pivotal role in medical education assessment. The Advanced Clinical Skills (ACS) OSCE examines clinical skills in psychiatry, general practice, obstetrics and gynaecology and paediatrics. This study examined if the 2020 ACS OSCE for fourth year medical students attending the National University of Ireland, Galway, was associated with any significant result differences compared to the equivalent 2019 OSCE. Additionally, we assessed students' satisfaction and explored any organisational difficulties in conducting a face-to-face OSCE during the COVID-19 pandemic.

MATERIALS AND METHODS

This study compared anonymised data between the 2019 and 2020 ACS OSCEs and analysed anonymised student feedback pertaining to the modified 2020 OSCE.

RESULTS

The mean total ACS OSCE result achieved in 2020 was statistically higher compared to the 2019 OSCE [62.95% (SD = 6.21) v. 59.35% (SD = 5.54), t = 6.092, p < 0.01], with higher marks noted in psychiatry (p = 0.001), paediatrics (p = 0.001) and general practice (p < 0.001) with more students attaining honours grades (χ2 = 27.257, df = 3, p < 0.001). No difference in failure rates were found. Students reported feeling safe performing the 2020 OSCE (89.2%), but some expressed face-mask wearing impeded their communication skills (47.8%).

CONCLUSION

This study demonstrates that conducting a face-to-face OSCE during the pandemic is feasible and associated with positive student feedback. Exam validity has been demonstrated as there was no difference in the overall pass rate.

Impact of familiarity with the format of the exam on performance in the OSCE of undergraduate medical students - an interventional study

BACKGROUND

Assessments, such as summative structured examinations, aim to verify whether students have acquired the necessary competencies. It is important to familiarize students with the examination format prior to the assessment to ensure that true competency is measured. However, it is unclear whether students can demonstrate their true potential or possibly perform less effectively due to the unfamiliar examination format. Hence, we questioned whether a 10-min active familiarization in the form of simulation improved medical students´ OSCE performance. Next, we wanted to elucidate whether the effect depends on whether the familiarization procedure is active or passive.

METHODS

We implemented an intervention consisting of a 10-min active simulation to prepare the students for the OSCE setting. We compared the impact of this intervention on performance to no intervention in 5th-year medical students (n = 1284) from 2018 until 2022. Recently, a passive lecture, in which the OSCE setting is explained without active participation of the students, was introduced as a comparator group. Students who participated in neither the intervention nor the passive lecture group formed the control group. The OSCE performance between the groups and the impact of gender was assessed using X2, nonparametric tests and regression analysis (total n = 362).

RESULTS

We found that active familiarization of students (n = 188) yields significantly better performance compared to the passive comparator (Cohen´s d = 0.857, p < 0.001, n = 52) and control group (Cohen´s d = 0.473, p < 0.001, n = 122). In multivariate regression analysis, active intervention remained the only significant variable with a 2.945-fold increase in the probability of passing the exam (p = 0.018).

CONCLUSIONS

A short 10-min active intervention to familiarize students with the OSCE setting significantly improved student performance. We suggest that curricula should include simulations on the exam setting in addition to courses that increase knowledge or skills to mitigate the negative effect of nonfamiliarity with the OSCE exam setting on the students.

Comparison of Peyton's Four-Step Approach With the Conventional Bedside Technique in Teaching Clinical Examination Skills to Medical Students

BACKGROUND

 Conventional bedside teaching (CBT) is an integral and classical method for imparting clinical skills to undergraduates in medical schools. It is a traditionally successful approach, especially when it comes to imparting patient-doctor relationship skills and knowledge on clinical management. Peyton's four-step approach (PFSA) is one of the newer structured instructional approaches for teaching-learning, especially for imparting procedural and complex psychomotor skills. The present study compares the application of PFSA in teaching complex systemic examination skills to the CBT technique in teaching the same skill to MBBS students. The impact of the acquisition of this examination skill was assessed statistically to compare PFSA and CBT methodologies.

METHODOLOGY

 The target population was MBBS (Bachelor of Medicine and Bachelor of Surgery) students; for this study, the phase II MBBS students were considered as the study population since they were relatively naïve to clinical bedside examination skills. Students were allotted groups and they were taught clinical skills through CBT and PFSA separately. Using the OpenEpi toolkit Version 3 open-source sample size calculator for comparing two means, the sample size was 30 students in each group. The students were assessed for their competency and were also made to fill out a feedback questionnaire to compare the two methods of education dispensing.  Results: The results of this study showed that PFSA is definitely suitable for teaching clinical examination skills. The acquisition of skills was found non-inferior to CBT while the retention of these skills was found to be equally good or even superior with PFSA than with CBT.  Conclusion: PFSA has already been proven to be a good teaching method for the acquisition of complex procedural skills. This study expands the role of PFSA in teaching clinical bedside examination skills to medical students. Further large-scale studies may clarify the learning impact and outcomes of PFSA in clinical bedside teaching.

Multiple stations in an integrated design of cardiovascular system morphology for medical undergraduate

The morphology knowledge is essential for clinical, diagnostic and surgical practice in medicine. However, it is a great challenge teaching this science in an integrated curriculum, since it has the need of active methods associated with technology, in a total impaired workload. Therefore, this work described an educational design of multiple practice stations in order to teach morphology of the cardiovascular system in the undergraduate medical education. This activity was conducted in the Multicampi School of Medical Sciences of Federal University of Rio Grande do Norte (UFRN) in Caicó/Brazil. Four anatomy stations and four embryology and histology stations of heart and blood vessels were drawn up. Anatomy stations approached cardiac cycle and cardiac valves (1 A); mediastinum and pericardium (2 A); internal morphology and heart conduction system (3 A) and external morphology of heart (4 A). Whereas embryology and histology stations approached embryogenesis of the heart (1B); cardiac microanatomy and fetal circulation (2B); and vascular microanatomy (3B;4B). All the stations emphasized cognitive and psychomotor fields associated with clinical correlations to active application of morphology knowledge. The multiple stations were considered as an organized, productive, enlightening and riveting alternative to medical students and teaching staff. This innovative experience integrated and energized the three areas of morphological sciences, resignifying the teaching and learning of cardiovascular system morphology.

Development of a low-cost congenital abdominal wall defect simulator (wall-go) for undergraduate medical education: a validation study

BACKGROUND

Congenital Anomalies were responsible for 303,000 deaths in the neonatal period, according to the WHO, they are among the world's top 20 causes of morbidity and mortality. Expensive simulators demonstrate several diseases, but few are related to congenital anomalies. This study aims to develop, validate, and evaluate low-cost simulator models (WALL-GO) of the most common abdominal wall defects, gastroschisis, and omphalocele, to enable diagnosis through an accessible tool with study value and amenable to replication.

METHODS

Market research was conducted to find materials to build low-cost models. The researchers built the model and underwent validation assessment of the selected experts who scored five or more in the adapted Fehring criteria. The experts were assessed through a 5-point Likert scale to 7 statements (S1-7). Statements were assigned values according to relevance in face and transfer validities. Concomitantly, the model was also evaluated by students from 1st to 5th year with the same instruments. Content Validity Indexes (CVIs) were considered validated between groups with concordance greater than 90%. Text feedback was also collected. Each statement was subjected to Fisher's Exact Test.

RESULTS

Gastroschisis and omphalocele model costs were US $15 and US $27, respectively. In total, there were 105 simulator evaluators. 15 experts were selected. Of the 90 students, there were 16 (1st year), 22 (2nd), 16 (3rd), 22 (4th), and 14 (5th). Students and experts obtained CVI = 96.4% and 94.6%, respectively. The CVIs of each statement were not significantly different between groups (p < 0,05).

CONCLUSIONS

The WALL-GO models are suitable for use and replicable at a manufacturable low cost. Mannequins with abdominal wall defects are helpful in learning to diagnose and can be applied in teaching and training health professionals in developing and low-income countries.

Structures, processes and outcomes of objective structured clinical examinations in dental education during the COVID-19 pandemic: A scoping review

INTRODUCTION

Objective structured clinical examinations (OSCEs) are an essential examination tool within undergraduate dental education. Fear of spread of the COVID-19 virus led to dental institutions exploring alternative means of conducting OSCEs. The aim of this scoping review was to investigate what structures, processes and outcomes of dental OSCEs were reported during the COVID-19 pandemic.

MATERIALS AND METHODS

This scoping review was conducted and reported adhering to the Preferred Reporting Items for Systematic Reviews and Meta-analyses extension for scoping review guidelines (PRISMA-ScR). Published literature was identified through a systematic search of PubMed, Embase, Cumulative Index to Nursing and Allied Health Literature (CINAHL), Education Resources Information Center (Eric), ProQuest and Google Scholar. Identified articles were independently reviewed by two authors (KS, AD), followed by synthesis in terms of the reported structures, processes and outcomes. Articles reporting cancellation or rescheduling were also included, extracting data on reasons and any suggestions/recommendations.

RESULTS

The search yielded a total of 290 studies of which 239 sources were excluded after removal of duplicates, leaving 51 studies for title and abstract evaluation. Thirty-four articles were excluded as they did not report on the topic of interest, leaving 17 for full-text evaluation, of which nine were analysed according to the pre-set themes. All dental OSCEs taking place (n = 6) were conducted online whilst the remaining (n = 3) were either cancelled or rescheduled. Data on structures reported specific online videoconferencing software used and provision of staff and student training. Processes on the execution of online OSCEs varied significantly from one study to the other, providing rich data on how dental institutions may carry out such assessments tailored to their need. Information regarding outcomes was sparse, as little attention was paid to the results of the students compared to pre-pandemic, lacking investigation into reliability and validity of online dental OSCEs.

CONCLUSION

Dental OSCEs could be conducted online implementing well-planned structures and processes; however, further evidence is needed to prove its reliability and validity based on outcomes. Dental institutions may need to consider alternative methods to assess practical competencies if online OSCEs are to take place.

Comparability of Objective Structured Clinical Examinations (OSCEs) and Written Tests for Assessing Medical School Students' Competencies: A Scoping Review

Objective Structured Clinical Examinations (OSCEs) and written tests are commonly used to assess health professional students, but it remains unclear whether the additional human resources and expenses required for OSCEs, both in-person and online, are worthwhile for assessing competencies. This scoping review summarized literature identified by searching MEDLINE and EMBASE comparing 1) OSCEs and written tests and 2) in-person and online OSCEs, for assessing health professional trainees' competencies. For Q1, 21 studies satisfied inclusion criteria. The most examined health profession was medical trainees (19, 90.5%), the comparison was most frequently OSCEs versus multiple-choice questions (MCQs) (18, 85.7%), and 18 (87.5%) examined the same competency domain. Most (77.5%) total score correlation coefficients between testing methods were weak (r < 0.40). For Q2, 13 articles were included. In-person and online OSCEs were most used for medical trainees (9, 69.2%), checklists were the most prevalent evaluation scheme (7, 63.6%), and 14/17 overall score comparisons were not statistically significantly different. Generally low correlations exist between MCQ and OSCE scores, providing insufficient evidence as to whether OSCEs provide sufficient value to be worth their additional cost. Online OSCEs may be a viable alternative to in-person OSCEs for certain competencies where technical challenges can be met.

Can trauma leagues contribute to better cognitive performance and technical skills of medical students? The experience of the Unicamp trauma league

PURPOSE

Trauma leagues (TLs) are extracurricular programs that offer medical students supervised exposure to trauma and acute care surgery, mentorship, and participation in other academic activities. TLs are fully approved medical schools, and currently, over 100 TLs exist in Brazil. We hypothesized that the performance/competence of medical students who participated in TLs was superior compared to non-participants. This study evaluated and compared the cognitive performance and technical skills of the two groups.

METHODS

This retrospective cohort study evaluated the performance of TL medical students to non-TL alumni from 2005 to 2017, using the students' academic performance coefficient, Clinical Competence Assessment, and Progress Test results. SigmaPlot 12.0 software was used to perform statistical analyses, including Mann-Whitney comparison tests and the Kruskal-Wallis test to confirm the data.

RESULTS

Of the 1366 medical students who graduated from a Brazilian university, 966 were included, with 17.9% having participated in TL. Compared to non-TL participants, TL students demonstrated better cognitive performance according to the performance coefficient (p = 0.017) and Progress Test result (p < 0.001), and higher achievement in the Clinical Competence Assessment (p < 0.001).

CONCLUSION

The academic performance of TL students was superior to that of non-TL students at the University of Campinas (Unicamp), suggesting a positive impact of TL in the preparation of future doctors. The study findings suggest that participation in TL at Unicamp was beneficial in preparing better doctors and should be considered by medical schools worldwide.

EVIDENCE LEVEL

II (Retrospective cohort).

Assessment of Perception of Dental Students to OSCE Exams: A Cross-Sectional Study

Aims and Objective

To assess the knowledge as well as attitude of dental students to OSCE exams.

Materials and Methods

With the aim of evaluating the knowledge and attitude of dental students to OSCE exams, the present study was planned and it consisted of total 1000 dental students (Third year, Final year, and Interns) who have taken the OSCE examinations. The survey included a questionnaire in addition to a subsection on participants' demographic information. In addition, a 3-point scale was used to rate the OSCE's impartiality, complexity, education level, as well as favored frequency of usage in comparison with various evaluation formats.

Results

562 were males while the remaining 438 were females. While evaluating the student's perception, 36.3 percent of the students agreed that OSCE examination gave precise measure of clinical dental skills. 23.1 percent of the students said OSCE was uniform in terms of standardization, while 25 percent of the students agreed that OSCE score was independent of personality, ethnicity, and gender.

Conclusion

To recapitulate, the outcomes of this research gave rise to the notion that the OSCE represents a valid as well as objective evaluation tool for clinical abilities.

Comparison of Virtual Simulations with Peers versus Actors as Standardized Patients on Virtual OSCE Scores

OBJECTIVE

This study's primary aim is to assess the use of different types of standardized patients (SPs) during formative simulation activities on summative objective structured clinical exams (OSCE) in a PharmD curriculum.

METHODS

Randomized-controlled study with first-year pharmacy students in a Pharmacist Patient Care Lab (PCL) course. Students were randomized into groups with either hired actors or their peers as SPs for virtual simulation activities. All students then completed a virtual teaching OSCE (TOSCE) and virtual OSCE. A mixed effects analysis was done to compare TOSCE and OSCE scores between the two groups.

RESULTS

There were no significant differences between the two groups in their TOSCE or OSCE scores for the analytical and global rubrics.

CONCLUSION

This study demonstrates that peers may be as effective as having hired actors as SPs in preparing students for virtual skills exams.

Evidence for commonly used teaching, learning and assessment methods in contact lens clinical skills education

INTRODUCTION

Evidence based practice is now an important part of healthcare education. The aim of this narrative literature review was to determine what evidence exists on the efficacy of commonly used teaching and learning and assessment methods in the realm of contact lens skills education (CLE) in order to provide insights into best practice. A summary of the global regulation and provision of postgraduate learning and continuing professional development in CLE is included.

METHOD

An expert panel of educators was recruited and completed a literature review of current evidence of teaching and learning and assessment methods in healthcare training, with an emphasis on health care, general optometry and CLE.

RESULTS

No direct evidence of benefit of teaching and learning and assessment methods in CLE were found. There was evidence for the benefit of some teaching and learning and assessment methods in other disciplines that could be transferable to CLE and could help students meet the intended learning outcomes. There was evidence that the following teaching and learning methods helped health-care and general optometry students meet the intended learning outcomes; clinical teaching and learning, flipped classrooms, clinical skills videos and clerkships. For assessment these methods were; essays, case presentations, objective structured clinical examinations, self-assessment and formative assessment. There was no evidence that the following teaching and learning methods helped health-care and general optometry students meet the intended learning outcomes; journal clubs and case discussions. Nor was any evidence found for the following assessment methods; multiple-choice questions, oral examinations, objective structured practical examinations, holistic assessment, and summative assessment.

CONCLUSION

Investigation into the efficacy of common teaching and learning and assessment methods in CLE are required and would be beneficial for the entire community of contact lens educators, and other disciplines that wish to adapt this approach of evidence-based teaching.

Formative objective structured clinical examination with immediate feedback improves surgical clerks' self-confidence and clinical competence

OBJECTIVE

Clerkship is crucial for fourth-year medical students before entering the clinical environment. However, lack of confidence impairs clerks' performance during the clinical rotation. We assess the impact of formative Objective Structured Clinical Examination (OSCE) with immediate feedback on surgical clerks' self-confidence and clinical competence.

METHODS

This is a prospective randomized controlled study. Thirty-eight fourth-year medical students starting their surgical clerkship were randomly divided into the control group (n = 19) and the OSCE group (n = 19), where an extra 6-station formative OSCE was given prior to the surgical rotation with immediate feedback on the participant's performance. Self-confidence assessment (SCA) was collected from each participant before, right after the formative OSCE and one month later. Clinical competence was assessed using a mini-clinical evaluation exercise (mini-CEX) with a case of acute abdominal pain and direct observation of procedural skills (DOPS) with incision and suture one month later.

RESULTS

The SCAs were significantly improved in the OSCE group right after the training, and a month later, compared to the control group. The mini-CEX score was significantly higher in the OSCE group compared to the control group, but not the DOPS score of incision and suture.

CONCLUSION

The formative OSCE with immediate feedback could significantly enhance surgical clerks' self-confidence and their clinical competence when taking the history, performing the physical examination, and in clinical reasoning; however, the formative OSCE did not improve their dexterity in performing the procedural skills.

Objective structured clinical examination in basic thoracic ultrasound: a European study of validity evidence

BACKGROUND

Basic thoracic ultrasound is being used more frequently by clinicians in several settings due to its high diagnostic accuracy for many common causes of respiratory failure and dyspnoea. However, ultrasound examinations are operator-dependent, and sufficient competences are needed to obtain high sensitivity and specificity of examinations. Additionally, it is crucial for ultrasound operators to perceive the competence to interpret the images and integrate them into the patient history and other examinations. This study aims to explore and gather validity evidence for an objective structured clinical examination test of basic thoracic ultrasound competences and establish a pass/fail score.

METHODS

An expert panel created the test which included two theoretical and five practical stations representing cases with different diagnoses that cause respiratory symptoms and which are possible to establish by basic thoracic ultrasound. Twenty-five participants with different levels of experience in basic thoracic ultrasound completed the test. Data of the test scores were used for item analysis, and exploring validity evidence was done according to Messick's framework which is recommended. The contrasting groups' standard setting method was used to establish a pass/fail score.

RESULTS

The summarised internal consistency reliability was high with a Cronbach's alpha of 0.87. The novice group (n = 4) had a mean test score of 42 ± 10.1 points, the intermediate group (n = 8) scored 79.1 ± 8.1 points, and the experienced group (n = 13) 89.0 ± 6.2 points (one-way ANOVA, p < 0.001). A pass/fail score of 71 points was thus derived (maximum test score = 105 points).

CONCLUSION

We developed a test for the assessment of clinical competences in basic thoracic ultrasound with solid validity evidence, and a pass/fail standard with no false positives or false negatives.

Part 2: Assessment of a virtual vs. onsite interview experience from the interviewer perspective

INTRODUCTION

The University of Texas at Austin College of Pharmacy transitioned from onsite interviews to virtual interviews in fall 2020. There is limited literature on whether the virtual format impacts an interviewer's assessment of a candidate. This study examined interviewer ability to assess candidates and barriers to participation.

METHODS

During the virtual interview process, interviewers utilized a modified multiple mini interview (mMMI) format to evaluate prospective college of pharmacy students. An 18-item survey was emailed to 62 interviewers from the 2020-2021 cycle. Virtual mMMI scores were compared to the previous year's onsite MMI scores. Descriptive statistics and thematic analysis were used to assess the data.

RESULTS

The response rate to the survey was 53% (33/62), and 59% of interviewers preferred virtual interviews to in-person. Interviewers stated that there were fewer barriers to participation, increased comfort, and more time with applicants during virtual interviews. For six of the nine attributes evaluated, ≥ 90% of interviewers reported that they were able to assess applicants as well as they did in person. When comparing virtual and onsite MMI scores, seven of nine attributes were statistically significantly higher in the virtual cohort than onsite.

CONCLUSIONS

From the perspective of interviewers, the virtual interview decreased barriers to participation and still allowed the ability to assess the candidates. While offering a choice of interview setting to interviewers could increase accessibility, the statistically significant difference in MMI scores between virtual and onsite formats suggests that additional standardization is required to offer both formats simultaneously.

The Effects of a Scenario-Based Spiritual Care Course on Spiritual Care Competence among Clinical Nurses: A Quasi-Experimental Study

Across their lifespans, and in many clinical settings, patients have spiritual care needs. Many nurses lack competence related to providing spiritual care. Popular educational strategies, such as simulated educational programs and objective structured clinical examinations (OSCE), have not been widely adopted in nursing spiritual care education. The purpose of this study was to explore the effects of a scenario-based spiritual care course on spiritual care competence in nurses. This quasi-experimental study employed a repeated-measures pre-test/post-test design with assessments immediately before, immediately after, and 3 months post-intervention. Nurses providing direct patient care in diverse clinical settings were recruited from a large medical foundation in northern Taiwan. The intervention was a one day scenario-based spiritual care course and OSCE. The experimental group (n = 53) and controls (n = 85) were matched for their similar units, ages, working experience, and clinical ladder status. The Spiritual Care Competence Scale (SCCS), Spiritual Perspective Scale (SPS), Spiritual Care Perspective Scale-Revised (SCPS-R), and reflection logs were completed by both experimental and control groups. The Course Satisfaction Scale, OSCE Checklist, and Standardized Patient Feedback Scale (SPFS) were completed by the experimental group only. The experimental group had significantly higher SPS scores and self-evaluated SCCS scores, and lower SCPS-R scores (more positive spiritual perspectives), than controls at 3 months post-intervention. The experimental group showed significant within-subject effects at three time points on SPS scores, SCPS-R scores, and self-evaluated SCCS scores. Mean global performance of OSCE was 3.40 ± 0.91, and SP feedback indicated strengths and areas for improvement. In conclusion, the scenario-based spiritual care course effectively enhanced nurses' spiritual care competence, competence, and skills. Blended education techniques can therefore enhance nurses' ability to support patients with spiritual care needs.

Peer mentoring in nursing education: A concept analysis

AIM

This peer-mentoring concept analysis aimed to explore and define the concept of peer mentoring in nursing education and the impact it may have on nursing education.

BACKGROUND

A lack of literature exists explicitly defining peer mentoring in nursing education. The interchangeable use of processes, terms, and applications may contribute to the lack of consistency and obscurity of nursing education scholarship on peer mentoring. Since peer mentoring can be used to enhance student success, there is a need to clarify the concept of peer mentoring in nursing education for accuracy in future nursing education research.

DESIGN/REVIEW METHOD

Using Walker and Avant's framework, the authors identified defining attributes, antecedents, and consequences of peer mentoring.

DATA SOURCE

The literature search involved a combination of terms in the MEDLINE with Full Text, Academic Search Complete, Humanities International, ERIC, CINAHL, Health Source: Nursing/Academic Edition, and Consumer Health Complete-EBSCOhost databases.

RESULTS

This concept analysis revealed a consistent definition of peer mentoring for use in nursing education. The definition of peer mentoring in nursing education is a formal learning partnership between two individuals (mentor and mentee) with differing levels of nursing school experience. The peer mentor promotes a positive academic and emotionally supportive environment, decreasing stress and increasing the mentee's confidence and competency.

CONCLUSION

A shared definition and understanding of peer mentoring within nursing education can contribute to the consistent use of the concept in research and advance the scholarship of nursing education.

Effects of a sudden change in curriculum delivery mode in postgraduate clinical studies, following the COVID-19 pandemic

OBJECTIVE

To examine the effect of a change in the delivery mode of clinical neurology, a postgraduate subject at Macquarie University, following COVID-19 restrictions on face-to-face teaching.

METHODS

Participants were master of chiropractic students (n = 212) who completed 2019 or 2020 clinical neurology. The main outcome measure was a comparison of objective structured clinical examination (OSCE) marks between the 2019 and 2020 cohorts. The 2019 group underwent traditional, face-to-face learning, whereas the 2020 group were taught and examined remotely in session 1 but returned to campus in session 2. Descriptive analyses, between-group differences, and generalized linear models were performed.

RESULTS

Means for OSCE marks between the 2 groups were higher in the 2020 group in session 1 (p < .001). However, when students returned to campus in session 2, the means were significantly lower in the 2020 group compared with the 2019 group (p < .001). Generalized linear regression indicated that the web-based mode of delivery in 2020 might have had a significant impact on OSCE marks compared with their 2019 counterparts (p < .001).

CONCLUSION

The sudden change in the mode of delivery from face-to-face to remote learning and the change in testing methods in response to the global COVID-19 pandemic had a significant effect on clinical neurology student performance scores. The OSCE marks in the 2020 cohort for session 1 were higher than those in 2019. However, when the 2020 cohort returned to on-campus tutorials in session 2, their marks were lower than those of their 2019 counterparts.

Impact of an interdisciplinary curriculum for dental students (GeriDent-Cologne) on attitudes and awareness towards older people and geriatric conditions

INTRODUCTION

Training of dental students in the treatment of older patients must also consider the multiple chronic medical conditions that may be present. We developed an interdisciplinary gerodontology training curriculum for dental students (GeriDent-Cologne) to investigate its influence on the attitudes and expectancies of dental students towards the treatment of older patients.

MATERIALS AND METHODS

The curriculum (given to fourth-year students) comprised five clinical examination stations that simulated different medical conditions affecting older people. Students were able to experience related issues first hand, express their fears and problems concerning dental treatment in older people and, through discussion with the supervisor, develop strategies to overcome these difficulties and enhance treatment. An extended version of the Los Angeles Geriatric Attitudes Scale was given to fourth- and fifth-year students (who acted as controls) over one semester.

RESULTS

Dental students had a positive attitude towards older people, and their attitude did not change after attending the curriculum. However, participating in GeriDent-Cologne had a significant beneficial impact on the feeling of being prepared to treat older people, led to a significantly greater understanding of the impact of medical issues and resulted in a considerably higher level of geriatric clinical knowledge over time.

CONCLUSIONS

GeriDent-Cologne led to positive and semester-long changes in awareness and knowledge of specific geriatric problems that influence dental treatment. We recommend the curriculum as a blueprint for the module of senior dentistry implemented in the new German dental licencing regulations.

Development and Validation of a Three-Dimensional Printed Training Model to Teach Ultrasound-Guided Injections of the Cervical Articular Process Joints in Horses

Intra-articular injections are routinely performed to alleviate pain and inflammation associated with osteoarthritis in horses. Intra-articular injections require accurate needle placement to optimize clinical outcomes and minimize complications. This study's objectives were to develop and validate a three-dimensional (3D) printed model of an equine cervical articular process joint to teach ultrasound-guided injections. Five identical models of an equine cervical articular process joint were 3D printed and embedded in 10% ballistic gelatin. Experts' and novices' ability to successfully insert a needle into the joint space of the model using ultrasound guidance was assessed and graded using an objective structured clinical examination (OSCE). Scores from experts and novices were compared to evaluate the construct validity of the model. Participants also answered a survey assessing the face and content validity of the model. Experts required less time (22.51 seconds) for correct needle placement into the model joint space than novices (35.96 seconds); however, this difference was not significant (p = .53). Experts' median total OSCE score (14) was significantly higher (p = .03) than novices' (12), supporting the model's construct validity. Participants agreed on the face and content validity of the model by grading all survey questions greater than 7 on a 10-point Likert-type scale. In summary, we successfully developed a 3D printed model of an equine cervical articular process joint, partially demonstrated the construct validity of the model, and proved the face and content validity of this new training tool.

Effect of moderation on rubric criteria for inter-rater reliability in an objective structured clinical examination with real patients

Objectives: Objective structured clinical examinations (OSCEs) are used to assess clinical competence in medical education. Evaluations using video-recorded OSCEs are effective in reducing costs in terms of time and human resources. To improve inter-rater reliability, these evaluations undergo moderation in the form of a discussion between the raters to obtain consistency in grading according to the rubric criteria. We examined the effect of moderation related to the rubric criteria on the inter-rater reliability of a video-recorded OSCE with real patients.

Methods

Forty OSCE videos in which students performed range-of-motion tests at shoulder abduction on real patients were assessed by two raters. The two raters scored videos 1 to 10 without moderation and videos 11 to 40 with moderation each time. The inter-rater reliability of the OSCE was calculated using the weighted kappa coefficient.

Results

The mean scores of the weighted kappa coefficients were 0.49 for videos 1 to 10, 0.57 for videos 11 to 20, 0.66 for videos 21 to 30, and 0.82 for videos 31 to 40.

Conclusions

An assessment of video-recorded OSCEs was conducted with real patients in a real clinical setting. Repeated moderation improved the inter-rater reliability. This study suggests the effectiveness of moderation in OSCEs with real patients.

Implementation of a patient-teaching workshop to improve pharmacy students' competencies in patient-centered communication: a case report

BACKGROUND

The pharmacist-patient relationship has evolved over recent decades and the development of clinical pharmacy requires pharmacists to take patient-centered responsibilities. This requires a specific set of skills, such as patient-centered communication. Evaluation of students' competencies in patient-centered communication is challenging in academic settings and complementary assessment methods may be designed in order to overcome the limits of traditional preceptors' ratings or objective structured clinical examination (OSCE). There is increasing interest in a more active patient role in healthcare professional education and there are very few reports about patient-led education in pharmacies. Thus, the objective of this work was to implement a patient-teaching workshop and to assess its impact on pharmacy students' competencies in patient-centered communication.

METHODS

The workshop was developed in collaboration between four patients, a senior clinical pharmacist and a lecturer in education sciences and implemented in the hospital pharmacy residency program. The main course objective was acquiring the three competencies of the Calgary-Cambridge guide to the medical interview: (i) building a relationship, (ii) conducting structured interview and (iii) gathering information. The learning process integrated: working on participants' perception of pharmacists-patient communication, a first simulated interview, didactic learning and a second simulated interview. After simulated interviews, patients and peer residents assessed learner's performance with a competency chart and provided individual feedback. Assessment methods included comparisons between the first and second interview scores and an anonymous post-course survey.

RESULTS

Forty-seven residents and 19 patient teachers attended the session. Competency scores were higher after the second interview in all three competencies as rated by both patients (+ 25%) and peer residents (+ 29%). Residents expressed a high satisfaction and reported learning about conducting interviews and soft skills contributing to the development of a relationship with patients. "The involvement of patients" was expressed as most appreciated in the majority of the evaluation charts (87%) and the residents valued the importance of collaborative and interprofessional learning during the workshop. Three themes emerged: (1) patients' expertise, (2) reliability and (3) relationship, which underlined that the students estimated the patients were credible sources of information in this pedagogical context.

CONCLUSION

This patient-teaching approach improved patient-centered competencies of pharmacy residents and promoted partnership between patients and pharmacy students.

A New Immersive Virtual Reality Station for Cardiopulmonary Resuscitation Objective Structured Clinical Exam Evaluation

The Objective Structured Clinical Exam (OSCE) is an assessment tool used as a reliable method for clinical competence evaluation of students. This paper presents an investigation focused on the chain of survival, its related exploration, management, and technical skills, and how Virtual Reality (VR) can be used for the creation of immersive environments capable of evaluating students' performance while applying the correct protocols. In particular, the Cardiopulmonary Resuscitation (CPR) procedure is studied as an essential step in the development of the chain of survival. The paper also aims to highlight the limitations of traditional methods using mechanical mannequins and the benefits of the new approaches that involve the students in virtual, immersive, and dynamic environments. Furthermore, an immersive VR station is presented as a new technique for assessing CPR performance through objective data collection and posterior evaluation. A usability test was carried out with 33 clinicians and OSCE evaluators to test the viability of the presented scenario, reproducing conditions of a real examination. Results suggest that the environment is intuitive, quick, and easy to learn and could be used in clinical practice to improve CPR performance and OSCE evaluation.

A dataset of simulated patient-physician medical interviews with a focus on respiratory cases

Artificial Intelligence (AI) is playing a major role in medical education, diagnosis, and outbreak detection through Natural Language Processing (NLP), machine learning models and deep learning tools. However, in order to train AI to facilitate these medical fields, well-documented and accurate medical conversations are needed. The dataset presented covers a series of medical conversations in the format of Objective Structured Clinical Examinations (OSCE), with a focus on respiratory cases in audio format and corresponding text documents. These cases were simulated, recorded, transcribed, and manually corrected with the underlying aim of providing a comprehensive set of medical conversation data to the academic and industry community. Potential applications include speech recognition detection for speech-to-text errors, training NLP models to extract symptoms, detecting diseases, or for educational purposes, including training an avatar to converse with healthcare professional students as a standardized patient during clinical examinations. The application opportunities for the presented dataset are vast, given that this calibre of data is difficult to access and costly to develop.

Antibiotic prescription errors: the relationship with clinical competence in junior medical residents

BACKGROUND

A large portion of prescribing errors can be attributed to deficiencies in medication knowledge. These errors are preventable and most often occur at the time of prescription. Antimicrobials are the drug class most common incorrectly prescribed.

OBJECTIVE

To characterize the relationship between clinical competence and antibiotic prescription errors. We also investigated the frequency and severity of antibiotic prescription errors to identify items and attributes of clinical competence which are correlated with the antibiotic prescription error ratio.

METHOD

A cross-sectional study was applied to assess clinical competence of junior medical residents in two reference academic hospitals and a regional hospital in Mexico City. It was conducted during February 2019. We used an infectious disease Objective Structured Clinical Examination (OSCE) to assess clinical competence and a measure of frequency, and severity of antibiotic prescription errors.

RESULTS

The number of eligible participants was ~ 255 (hospital meeting attendance), and the number of residents in this study were 51 (~ 20%), 31 were female (60.8%). The mean OSCE score was 0.692 ± 0.073. The inter-item (Cronbach's alpha = 0.927) and inter-station internal consistency was adequate (Cronbach's alpha = 0.774). The G coefficient in generalizability theory analysis was 0.84. The antibiotic prescription error ratio was 45.1% ± 7%. The most frequent category of severity of antibiotic prescription errors was category E (errors that may contribute to or result in temporary harm to the patient and require intervention), 235 (65.2%). We observed a negative and significant correlation between clinical competence and antibiotic prescription errors (r = -0.33, p < 0.05, CI95% -0.57 to -0.07), which remained significant after controlling for the effect of gender and time since graduation from medical school (r = -0.39, p < 0.01, CI95% -0.625 to -0.118). Using exploratory factor analysis we identified two factors, which explained 69% of the variance in clinical competence, factor 1 evaluated socio-clinical skills and factor 2 evaluated diagnostic-therapeutic skills. Factor 2 was correlated with antibiotic prescription error ratio (r = -0.536, p < 0.001).

CONCLUSIONS

We observed a negative correlation between clinical competence and antibiotic prescription error ratio in graduated physicians who have been accepted in a medical specialty. The therapeutic plan, which is a component of the clinical competence score, and the prescription skills had a negative correlation with antibiotic prescription errors. The most frequent errors in antibiotic prescriptions would require a second intervention.

Flipped classroom and the use of role-play in the COVID-19 pandemic

Aim: In Brazil, COVID-19 remains one of the epicenters of the pandemic, thus, presential educational activities are suspended. The study aimed to describe the approach of flipped classroom performed to provide social distancing and to maintain teaching activities during COVID-19-pandemic. Methods: Flipped Classroom was chosen to be associated with the role-play technique as a central methodology. To bring students closer to activities performed in the non-pandemic period, some actions and assessments were also role-played by Professor simulating patients in virtual meetings. Results: Although the changes in the format of the lessons, our approaches stimulated the students maintained the high frequency of students in the activities (almost 100%) providing continuity activities. Conclusions: This report can reinforce that meaningful learning can be taught by using a virtual/remote approach. However, the potential positive results observed cannot be used as an excuse to maintain remote teaching activities with the objective of cost-cutting by the institutions.

Teaching and assessment of clinical diagnostic reasoning in medical students

BACKGROUND

Teaching diagnostic reasoning and giving feedback has an important role in medical education. Clinicians who teach may recognise errors, but be unfamiliar with the terminology used to describe them, leading to a lack of consistent and useful student feedback.

OBJECTIVE

This prospective project evaluation study aimed to develop an examiner training package regarding errors in diagnostic reasoning, utilising consistent language and feedback tool, and report on diagnostic reasoning errors in second year medical students over the transition from preclinical to early clinical training at objective structured clinical exams (OSCEs).

RESULTS

Likert questionnaire regarding examining, assessment and feedback pre- and post-training showed improvement in all measures, including examiner feedback confidence post training (p < .001). Students (n = 235) within the cohort were examined at the first preclinical OSCE 12 weeks into the teaching year and 236 students at the end of year OSCE. A range of 0-6 diagnostic reasoning errors were reported for individual students. When comparing mean history station scores at the preclinical OSCE for students who were observed to have diagnostic reasoning errors, students with 'poor pattern recognition' had a 4.2% lower score than those without this error type (p = .04, 95% CI of difference .14, 8.32), while those with 'unfocused data collection' error had a station score 7.7% lower than those without this error (p < .001, 95% CI of difference 3.50, 11.99). At the end of teaching year clinical OSCE, all common error types were associated with poorer performance. Error pattern shifted through the two longitudinal assessments, resulting in 'poor pattern recognition' having reduced and 'too narrow' and 'premature closure' increased rates.

CONCLUSIONS

Incorporating the identification and feedback of common diagnostic reasoning errors into existing clinical assessments was feasible and easy to implement. Understanding, identifying and providing consistent feedback on common errors assists educators and could guide curriculum design.

Curricular changes and interim posts during Covid-19: graduates' perspectives

BACKGROUND

During the COVID-19 pandemic UK medical schools facilitated the early graduation of their final-year medical students to 'Foundation interim Year 1 (FiY1) doctors' through amendments made to curricula and final assessment. Such changes gave opportunity for evaluation. This study therefore aimed to explore 1) graduate perspective on the implementation of FiY1 and 2) how changes to course structures have affected self-reported preparedness for work.

METHODS

Questionnaire surveys using Likert scale and free-text responses (n = 45), and semi-structured interviews (n = 7) were conducted with FiY1s from two UK medical schools contrasting in the amendments made to course structures. Data were analysed using quantitative methods and thematic analysis; 44% (n = 20) of respondents believed that governing health bodies had not communicated sufficiently prior to starting work.

RESULTS

Graduates who had sat modified practical and written examinations reported 'legitimacy' and feeling more prepared compared to having not sat examinations (practical 100%, n = 17; written 88.3%, n = 15). Graduates from both schools agreed that carrying out assistantships as originally scheduled would have made them feel more prepared (91.1%, n = 41).

CONCLUSIONS

The implementation of FiY1 was largely well received by graduates yet assistantship programmes may fulfil a similar role in normal times. Medical schools and governing bodies must ensure effective communication channels exist with students in order to better prepare them for their first posts, especially in times of crisis. Additionally, final examinations contribute to feelings of preparedness for work and instil a sense of legitimacy, a finding which is relevant to working within the current programmatic assessment structure.

A peer-led mock OSCE improves student confidence for summative OSCE assessments in a traditional medical course

BACKGROUND

Objective structured clinical examinations (OSCEs) can induce states of stress and anxiety in students, possibly negatively impacting student performance. Students in traditional-style medical courses especially have limited clinical exposure prior to their first clinical examination. We describe the design and implementation of a near-peer-led mock OSCE for fourth year medical students, and its impact on students' self-rated confidence.

METHODS

An OSCE training programme was designed by penultimate and final year medical students. It involved the recruitment of 12 peer tutors to facilitate the running and feedback of a mock OSCE to 82 fourth year (and second year graduate) medical students delivered over 2 days, prior to their end-of-year exam. Students completed a post activity questionnaire to assess the quality and perceived benefits of the mock OSCE.

RESULTS

A total of 78 students completed the survey, giving a response rate of 92.8%. 100% of respondents felt the OSCE was well run, well-structured and highlighted areas of weakness in their knowledge and skills. Students reported the OSCE significantly improved their confidence for their summative OSCE. The main themes from the student comments were feeling empowered and feeling that the mock OSCE was realistic.

CONCLUSIONS

The near-peer-led OSCE significantly improved student confidence for OSCEs for fourth year undergraduate and second year graduate students studying a traditional medical course and was well reviewed by students.

Usability and preference of electronic vs. paper and pencil OSCE checklists by examiners and influence of checklist type on missed ratings in the Swiss Federal Licensing Exam

BACKGROUND

Only a few studies with small sample sizes have compared electronic Objective Structured Clinical Examination (OSCE) rating checklists with traditional paper-based OSCE rating checklists. In this study, the examiner-perceived usability and preference for type of OSCE checklist (electronic vs. paper based) were compared, and the influence of OSCE checklist type on missed ratings was determined, for the Swiss Federal Licensing Examination in clinical skills for human medicine.

METHODS

All examiners in the Swiss Federal Licensing Examination in clinical skills for human medicine were invited over two subsequent years to evaluate the OSCE checklist type they had worked with during the examination. This was based on a questionnaire with 14 closed questions (i.e., demographic, checklist-type experience, perceived usability, checklist type preference). Furthermore, the numbers of missed ratings for the paper-based checklist were recorded.

RESULTS

The data from these examiners (n=377) with experience of both OSCE checklist types were analyzed. The electronic OSCE checklist was rated significantly higher on all usability aspects (i.e., ease of use, candidate rating and error correction, clarity, distraction using the checklist, overall satisfaction), except for the speed of registering comments (no significant difference). The majority of the examiners in both years (2014: 54.5%, n=60, 2015: 89.8%, n=230) reported preference for working with the electronic OSCE checklist in the future. Missed ratings were seen for 14.2% of the paper-based OSCE checklists, which were prevented with the electronic OSCE checklists.

CONCLUSIONS

Electronic OSCE checklists were rated significantly more user-friendly and were preferred over paper-based OSCE checklists by a broad national sample of examiners, supporting previous results from faculty-level examinations. Furthermore, missed ratings were prevented with the electronic OSCE checklists. Overall, the use of electronic OSCE checklists is therefore advisable.

Experiences and opinions of multi-professional non-medical oncology prescribers on post-qualification training: a qualitative study

Background: Within the UK, a non-medical prescriber is a non-medical healthcare professional who has undertaken post-registration training to gain prescribing rights. Lack of post-qualification NMP training has previously been identified as a barrier to the development of oncology non-medical prescribing practice. Aim: To explore the experiences and opinions of multi-professional non-medical oncology prescribers on post-qualification training. Method: Nine out of 30 oncology non-medical prescribers (three nurses, three pharmacists and three radiographers) from a single cancer centre in Wales, were selected from a study site NMP database using randomisation sampling within Microsoft® Excel. Participants were interviewed using a validated and piloted semi-structured interview design on the topic of post-qualification training for non-medical prescribers. Participants were invited via organisational email. Interviews were audio-recorded and transcribed verbatim. Anonymised data were thematically analysed aided by NVivo® software. Results: Main themes identified: experience related to training, competency, support and training methods. Competency assessment methods discussed were the annual non-medical prescriber appraisal, peer review and a line manager’s overarching appraisal. Support requirements identified included greater consultant input to help non-medical prescribers identify training and peer support opportunities. Organisational support was requested regarding regular study leave and governance around clinical judgement and errors. The need for regular structured in-house training related to non-medical prescriber’s level of experience was identified. Conclusion: Development of organisation-led governance strategies and in-house training programmes will support training equity for all non-medical prescribers within the organisation.

Comparison of student performance in therapeutics and communications courses to outcomes of objective structured clinical examinations: A retrospective analysis

INTRODUCTION

The objective was to examine the association between course grades and objective structured clinical examination (OSCE) performances in a pharmacy student cohort.

METHODS

A retrospective analysis of demographics, final grades in the Therapeutics I through IV and Communications courses, and multiple OSCE scores (therapeutic knowledge and general communication skills [GCS]) was performed for two cohorts.

RESULTS

Female students scored higher than male students on the warfarin OSCE therapeutic knowledge (P = .008) and GCS scores (P = .02). Age was inversely correlated with warfarin OSCE therapeutic knowledge score (P = .001). The warfarin OSCE therapeutic knowledge score was positively related to Therapeutics II final grades (P < .001). The Communications course final grade was positively correlated to the warfarin OSCE faculty-rated GCS score (P = .005). Therapeutics final grades were not significantly related to station scores of a multi-station OSCE (P > .05). The final regression model included gender and Therapeutics II final grade and explained 6% of the variance in warfarin OSCE therapeutic knowledge scores.

CONCLUSIONS

A number of significant associations were found between demographics, final course grades, and specific OSCE scores. A regression model was significant, but only explained a low percentage of the warfarin OSCE therapeutic knowledge score variance, suggesting other factors not evaluated had a greater effect on scores. This research suggests that OSCEs play an important role in demonstrating student competency in educational domains other than knowledge and brings forth new data suggesting that age and/or gender may influence OSCE performance.

Innovative Method to Digitize a Web-Based OSCE Evaluation System for Medical Students: A Cross-Sectional Study in University Hospital in Saudi Arabia

Purpose

Methods

Results

Conclusion

Quality management of objective structured clinical examinations: An integrative literature review

BACKGROUND

Objective Structured Clinical Examinations are widely adopted as a clinical assessment method in nurse education. Quality is an integral aspect of the design and implementation of OSCEs, facilitating their credibility and rigour. However, there is a dearth of literature regarding the management of the quality of Objective Structured Clinical Examinations.

OBJECTIVE

To summarise existing literature regarding best practices on the management of the quality of Objective Structured Clinical Examinations in health science education.

DESIGN

An integrative literature review.

DATA SOURCES

EBSCOhost, including CINAHL, eBook Collection, E-journals, ERIC, Health Source-Consumer Edition, Health Source-Nursing/Academic Edition, Humanities International Complete and MEDLINE, together with Cochrane Online, PubMed, Taylor & Francis Online and ScienceDirect, were searched, followed by hand searching of references as well as a manual search for grey literature, using Google.

REVIEW METHODS

Using predetermined inclusion and exclusion criteria, research documents (experimental, non-experimental, descriptive and qualitative studies), non-research documents (editorials, opinion letters), including grey literature, published between January 2010 and March 2021.

RESULTS

A total of thirteen (n = 13) studies were included in the review. A total of 22 quality measures were identified, which should be applied in the preparation and planning, implementation and evaluation phases of Objective Structured Clinical Examinations. The preparation and planning phase is crucial for the quality of Objective Structured Clinical Examinations, as more than half (n = 12) of the 22 quality measures were identified in this phase.

CONCLUSIONS

There is limited available recorded evidence that defines and describes the management of the quality of Objective Structured Clinical Examinations. Future research should focus on the context-specific use of psychometric analysis to ensure generation of valid, objective and consistent assessment during Objective Structured Clinical Examinations. More rigorous large-scale studies (such as quantitative or randomised controlled trials) regarding management of variances amongst examiner scores, impact of moderation on Objective Structured Clinical Examinations, and virtual Objective Structured Clinical Examinations should be conducted.

Digitizing Scoring Systems With Extended Online Feedback: A Novel Approach to Interactive Teaching and Learning in Formative OSCE

Objectives

Objective structured clinical examinations (OSCEs) are common for formative assessment. We developed an Online Smart Communicative Education System and aimed to explore the factors that affect the perceptions of both teachers and students for teaching and learning.

Methods and Analysis

A two-year cross-sectional cohort study was undertaken. The program includes three parts. Part I Pre-OSCE: an online flipped class in preparation for task-related knowledge and skills. Part II OSCE-day: 10 tasks in one track formative OSCE. Part III Post-OSCE: extended online feedback for participants with further questions after the exam and raters with more feedback after reviewing their performance online. Principal component analysis with varimax rotation was performed to analyze the perceptions of students and teachers to the Online System by means of questionnaires.

Results

Seventy-six pharmacy students (male 32.9%) took the exam and 24 raters (male, 25%) participated in the scoring during the OSCEs. The mean G coefficient was 0.88. Seventy-six questionnaires from the students were obtained for the analysis. Results explained the cumulative variance of 73.9% for component (1) “Effects of extended online feedback”: 40% and (2) “Facilitation of learning”: 33.9%. Thirty-nine questionnaires from the raters who experienced the Online System were obtained for the analysis (male 23.1%). Results explained a cumulative variance of 77.3% for component (1) “Effects of extended online feedback”: 36.6%, (2) “Facilitation of scoring and feedback”: 24.5%, and (3) “Feasibility of online platform”: 16.2%, respectively.

Conclusion

We demonstrated good reliability for digitizing the scoring system with educational support to facilitate teaching. “Effects of extended online feedback” was the major aspect in explaining the variance from the perceptions of students and raters by factor analysis. In comparison with traditional formative OSCEs, extended online feedback is a novel approach, which extends the process of learning and teaching among the learners and raters and overcomes the barriers of time limitation and distance.

Competency-Based Assessment Methods in Adult Cardiac Sonography: Students’ Perceptions and Clinical Evaluation

Objective:

This research examined the effects of multiple combined competency-based methods on sonography students’ perceptions of adult echocardiography training components. In addition, clinical preceptor evaluation scores were compared with faculty objective structured clinical examination (OSCE) scores.

Materials and Methods:

A quasi-experimental nonequivalent group research design was used to evaluate students enrolled in an adult cardiac Commission on Accreditation of Allied Health Education Programs (CAAHEP) accredited curriculum. Students’ perceptions pre and post multiple competency-based methods (formative assessment, OSCE, & simulation) intervention were recorded via course evaluations. Questions were analyzed individually using descriptive statistics and the Bonferroni correction. Students’ clinical evaluation and OSCE scores were analyzed using Spearman’s rank correlation.

Results:

The majority of students’ perceptions pre- and postintervention of multiple competency-based assessments demonstrated significant differences, or they primarily agreed that their echocardiography knowledge or skill set was enhanced. There was weak correlation between students’ clinical competency evaluation scores and OSCE scores—post implementation of multiple competency-based assessments, rs(37) = −.01, P = .93.

Conclusion:

These results suggest further evaluation of the credentialing process’s clinical assessment to ensure clinical competency.

An Abrupt Transition to Digital Teaching—Norwegian Medical Students and Their Experiences of Learning Output during the Initial Phase of the COVID-19 Lockdown

Norwegian universities closed almost all on-campus activities on the 12 March 2020 following a lockdown decision of the Norwegian government in response to the COVID-19 pandemic. Online and digital teaching became the primary method of teaching. The goal of this study was to investigate how the transition to digital education impacted on medical students enrolled at the University of Bergen (UiB). Key points were motivation, experience of learning outcomes, and fear of missing out on important learning. Using an online questionnaire, students were asked to evaluate the quality of both lectures and taught clinical skills and to elaborate on their experience of learning output, examination, and digital teaching. Answers from 230 students were included in the study. Opinions on the quality and quantity of lectures offered and their experience of learning output varied based on gender, seniority and the amount of time spent on part time jobs. Students at UiB were generally unhappy with the quality of teaching, especially lessons on clinical skills, although both positive and negative experiences were reported. Securing a satisfying offer of clinical teaching will be important to ensure and increase the student experience of learning output in the time ahead.

Twelve tips to organise a mock OSCE

The objective structured clinical examination (OSCE) is central to assessing clinical competence in undergraduate and postgraduate exams for medical and allied healthcare professions. A mock OSCE on the other hand is a simulation of the OSCE and a unique learning experience for the examinee. They benefit in a variety of ways; from enhancing their time management skills to receiving feedback that can improve their clinical skills. Unfortunately, opportunities to participate in simulated OSCEs remain limited. Reasons include difficulty in fulfilling organisational requirements and equipment-related costs. Mock OSCEs can be set up by undergraduate students or junior trainees for peers, without senior supervision or direct guidance. This article will discuss 12 tips regarding the arrangement of mock OSCEs to guide organisers, including accessing resources and establishing the content of the exam.

Objective structured clinical examination vs traditional clinical examination to evaluate students' clinical competence: A systematic review of nursing faculty and students' perceptions and experiences

BACKGROUND

Assessment of clinical competence of nursing students is an essential requirement in professional nursing education. This article summarizes the current published evidence indicating the nursing faculty and students' perceptions and experiences on benefits of OSCE over TCE.

METHODS

A systematic review was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) recommendations. Electronic databases including Scopus, Medline, Science Direct, CINAHL, EBSCO, PsychINFO, and Pubmed Central were used to identify relevant articles. The studies published between January 1, 2010 and December 31, 2020 and fourteen full-text articles that met all the inclusion criteria were included in the review.

RESULTS

The review identified five themes namely: a) Student's perception of OSCE; b) student's satisfaction regarding OSCE as an examination approach; c) students' perception of TCE as an examination approach; d) student's level of stress and anxiety towards OSCE vs TCE; and e) faculty member's perception and experience of OSCE.

CONCLUSION

We conclude that OSCE is a more credible assessment format to evaluate the clinical competence of undergraduate nursing students compared to the TCE method.

Perception of Students and Examiners about Objective Structured Clinical Examination in a Teaching Hospital in Ethiopia

INTRODUCTION

The objective structured clinical examination (OSCE) has become a standard assessment tool in undergraduate medical school training. It is considered an objective assessment of practical skill of students. OSCE is a resource demanding assessment method that can have numerous challenges. Comprehensive assessment of perception regarding OSCE can help identify areas that need improvement. The aim of this study was to assess the perception of students and examiners towards OSCE.

METHODS

A cross-sectional study was conducted on students and examiners undertaking OSCE from May 1 to July 30, 2021, using a structured questionnaire. Comparison of variables was done using Mann-Whitney U-Test and Chi-square test. P-value <0.05 was considered statistically significant.

RESULTS

A total of 141 students and 39 examiners participated in the study. The majority of the students and examiners had a positive response regarding the attributes, structure, organization and validity of OSCE. It was recommended to be used in future exams compared to other assessments by 38.3% of students and 51.3% of examiners. There were certain challenges reported by students and included stressfulness of the exam (51.1%), inadequate time (27.6%), and unsatisfactory orientation (30.5%). One-third of examiners considered it stressful, while 20.5% considered the time provided to be inadequate. Equipment to conduct the exam was considered inadequate by 39.1% and 56.4% of students and examiners, respectively. Around 80.1% of students recommended mock sessions and 23.1% of examiners did not have any prior training on OSCE.

CONCLUSION

An overall positive perception of OSCE by students and examiners was seen. Certain challenges that need improvements were identified. Continuing evaluation and refinement of OSCE by departments is needed. We recommend further wide-scale national evaluation of the OSCE examination system of medical students.