Medical Faculty's and Students' Perceptions toward Pediatric Electronic OSCE during the COVID-19 Pandemic in Saudi Arabia

Feasibility and reliability of the pandemic-adapted online-onsite hybrid graduation OSCE in Japan

Objective structured clinical examination (OSCE) is widely used to assess medical students' clinical skills. Virtual OSCEs were used in place of in-person OSCEs during the COVID-19 pandemic; however, their reliability is yet to be robustly analyzed. By applying generalizability (G) theory, this study aimed to evaluate the reliability of a hybrid OSCE, which admixed in-person and online methods, and gain insights into improving OSCEs' reliability. During the 2020-2021 hybrid OSCEs, one examinee, one rater, and a vinyl mannequin for physical examination participated onsite, and a standardized simulated patient (SP) for medical interviewing and another rater joined online in one virtual breakout room on an audiovisual conferencing system. G-coefficients and 95% confidence intervals of the borderline score, namely border zone (BZ), under the standard 6-station, 2-rater, and 6-item setting were calculated. G-coefficients of in-person (2017-2019) and hybrid OSCEs (2020-2021) under the standard setting were estimated to be 0.624, 0.770, 0.782, 0.759, and 0.823, respectively. The BZ scores were estimated to be 2.43-3.57, 2.55-3.45, 2.59-3.41, 2.59-3.41, and 2.51-3.49, respectively, in the score range from 1 to 6. Although hybrid OSCEs showed reliability comparable to in-person OSCEs, they need further improvement as a very high-stakes examination. In addition to increasing clinical vignettes, having more proficient online/on-demand raters and/or online SPs for medical interviews could improve the reliability of OSCEs. Reliability can also be ensured through supplementary examination and by increasing the number of online raters for a small number of students within the BZs.

Virtual Objective Structured Clinical Examination (OSCE) Training in the Pandemic Era: Feasibility, Satisfaction, and the Road Ahead

INTRODUCTION

Objective Structured Clinical Examinations (OSCEs) are essential assessments for evaluating the clinical competencies of medical students. The COVID-19 pandemic caused a significant disruption in medical education, prompting institutions to adopt virtual formats for academic activities. This study analyzes the feasibility, satisfaction, and experiences of pediatric board candidates and faculty during virtual or electronic OSCE (e-OSCE) training sessions using Zoom video communication (Zoom Video Communications, Inc., San Jose, USA).

METHODS

This is a post-event survey assessing the perceptions of faculty and candidates and the perceived advantages and obstacles of e-OSCE.

RESULTS

A total of 142 participants were invited to complete a post-event survey, and 105 (73.9%) completed the survey. There was equal gender representation. More than half of the participants were examiners. The overall satisfaction with the virtual e-OSCE was high, with a mean score of 4.7±0.67 out of 5. Most participants were likely to recommend e-OSCE to a friend or colleague (mean score 8.84±1.51/10). More faculty (66.1%) than candidates (40.8%) preferred e-OSCE (P=0.006).

CONCLUSION

Transitioning to virtual OSCE training during the pandemic proved feasible, with high satisfaction rates. Further research on virtual training for OSCE in medical education is recommended to optimize its implementation and outcomes.

An assessment of preclinical removable prosthodontics based on multiple-choice questions: Stakeholders' perceptions

PURPOSE

Item analysis of multiple-choice questions (MCQs) is an essential tool for identifying items that can be stored, revised, or discarded to build a quality MCQ bank. This study analyzed MCQs based on item analysis to develop a pool of valid and reliable items and investigate stakeholders' perceptions regarding MCQs in a written summative assessment (WSA) based on this item analysis.

METHODS

In this descriptive study, 55 questions each from 2016 to 2019 of WSA in preclinical removable prosthodontics for fourth-year undergraduate dentistry students were analyzed for item analysis. Items were categorized according to their difficulty index (DIF I) and discrimination index (DI). Students (2021-2022) were assessed using this question bank. Students' perceptions of and feedback from faculty members concerning this assessment were collected using a questionnaire with a five-point Likert scale.

RESULTS

Of 220 items when both indices (DIF I and DI) were combined, 144 (65.5%) were retained in the question bank, 66 (30%) required revision before incorporation into the question bank, and only 10 (4.5%) were discarded. The mean DIF I and DI values were 69% (standard deviation [Std.Dev] = 19) and 0.22 (Std.Dev = 0.16), respectively, for 220 MCQs. The mean scores from the questionnaire for students and feedback from faculty members ranged from 3.50 to 4.04 and from 4 to 5, respectively, indicating that stakeholders tended to agree and strongly agree, respectively, with the proposed statements.

CONCLUSION

This study assisted the prosthodontics department in creating a set of prevalidated questions with known difficulty and discrimination capacity.

Dental students' and faculty perception of online exams with e-invigilation in Cyprus

INTRODUCTION

During the recent pandemic, e-learning and e-assessment methods have been implemented worldwide, providing opportunities for further implementation in the dental curriculum. This study aims to research the dental students' and dental faculty's perceptions of online exams with e-invigilation.

MATERIALS AND METHODS

Online questionnaires were developed and delivered, after three semesters of online exams, to all students and faculty. Descriptive statistics were performed, and answers were grouped into Principal Components (PC) using Principal Component Analysis (PCA). Statistical significance was set at p < .05.

RESULTS

Two-hundred and sixty dental students (83.7%) and 24 dental faculty members (63.1%) answered the online questionnaires. PCA of students' responses revealed 4 PC, 'University support to students', 'Comparison between online and face-to-face exams', 'Preparation for the online exams' and 'Attitudes towards the technology used for the online exams'. PCA of faculty responses revealed 5 PC: 'Comparison between online and face-to-face exams', 'University support to faculty', 'Faculty attitudes towards the exam procedures', 'Human factors associated with the exam procedures' and 'Exam invigilation'. The overall satisfaction was high for both staff and students (higher for students and female staff). Students with previous experience in online exams scored more positively than first-year students. University support, process-related stress and e-invigilation were highlighted.

CONCLUSIONS

The overall satisfaction with the e-exams was high, despite the technical problems, time-consuming processes and related stress. University support (training, IT support and resources) and mock exams emerged as important elements of online exams, as was the e-invigilation, which students perceived as efficient and non-intrusive.

Medical education during the COVID-19 pandemic: What students missed and what they did not. A questionnaire-based cross-sectional study

Background

Medical education was and still is challenged by the COVID-19 pandemic, and several strategies were implemented by the universities worldwide in order to maintain a good level of education. The aim of this work is to point out how strategies adopted in a German university hospital reached students and how comfortable they felt with the proposed solutions in order to define future possibilities in modern teaching.

Methods

A questionnaire was answered by medical students at the end of the 8th and 10th semester in a German university hospital asking them about their perception of medical education during the pandemic as well as about strategies adopted by the faculty.

Results

A total of 92 out of 117 students answered the questionnaire (78.6% response rate). Students felt disadvantaged in their medical education because of the pandemic on a scale from 0 (not at all) to 10 (completely) (5.34±2.3, range 0-10 points), regardless of semester, gender, and whether they aimed at a surgical career or not. During the pandemic they missed practical exercises most (93.5%), followed by contact with other students (65.2%). Presence lessons were missed (28.3%) the least. Among the strategies offered to maintain education, recorded lessons were appreciated most, followed by skills labs. Live-stream lessons were considered less comfortable.

Conclusions

Several aspects of medical education were replaced satisfactorily during the pandemic, others need to be adapted in the future in order to meet the students' needs and expectations. Theoretical online education but not live stream lessons could be an option beyond COVID-19 as they are highly appreciated by students.

Medical Students' Perceptions Towards Online Teaching During the Covid-19 Pandemic: A Cross-Sectional Study from Saudi Arabia

Background

The Covid-19 has made a huge impact on higher education. Online teaching and learning became essential to deliver educational activities in all areas including medical education. In this study, we aimed to investigate medical students' perceptions on the role of online teaching and learning in facilitating medical education.

Material and Methods

A cross-sectional study using a self-administered online questionnaire was conducted. Students eligible were medical students across all years at Imam Abdulrahman Bin Faisal University, Saudi Arabia. Perceptions analysis was conducted using SPSS software.

Results

A total of 563 students participated in the study (prominent category female 64%, n = 361). There was a significant increase in the number of hours devoted to online learning during the pandemic. Live lectures/tutorials platform via zoom showed the highest rate of interaction compared to pre-recorded lectures and learning materials uploaded on blackboard. 50% of the students disagreed that online teaching is as effective as face-to-face teaching. The greatest perceived enjoyable aspect included the online accessibility of materials. Whereas the most frequent perceived barrier to online learning included internet connection. 17% of students reflected a poor understanding of scientific materials through online PBL. More than 50% of students revealed that online theoretical lectures are as good as classroom or better. Whereas the majority (70%) were unable to learn clinical skills online. The results indicated high impact on students' physical activities (80%). Impacts were higher on pre-clinical students' health and social life than on clinical students.

Conclusion

Our findings reported that during emergency situations due to the pandemic, online teaching enables the continuity of medical education and provides adequate efficiency. The use of live online platforms showed high level of interaction. However, some barriers need to be addressed especially at the clinical skills development level to maximize the benefit of online teaching and learning.

Supporting pharmacy students' preparation for an entry-to-practice OSCE using video cases

BACKGROUND AND PURPOSE

Objective structured clinical examinations (OSCEs) are widely used, including in licensure examinations. OSCEs assess diverse skills in a structured manner, but can be stressful for students and have a significant organisational burden for faculty. Case-based video OSCE preparation resources were developed for students preparing for a licensure OSCE. The study aimed to examine student engagement, compare scores awarded by students to performances at specific competence standards with faculty scores, and examine usability, usefulness, and acceptability of the video cases.

EDUCATIONAL ACTIVITY AND SETTING

Final year pharmacy students (n = 149) enrolled in an integrated master of pharmacy programme in Ireland were invited to participate. Six sets of recorded OSCE-based video cases were developed, comprising multiple recordings of the same case scenario, with each pitched at a different level of performance. Students watched and scored the video cases. Usability, usefulness, and acceptability were evaluated via questionnaire.

FINDINGS

One or more video cases were accessed by 70.5% of students. Score ratings awarded by the students, when compared to faculty ratings, showed an overall trend towards inter-rater agreement between students and faculty. Students felt positively in terms of the usability, usefulness, and acceptability of the videos.

SUMMARY

Video cases designed to support OSCE preparation for a high-stakes national pharmacy licensure examination were widely used by students and were perceived to be usable, useful, and acceptable. Such video cases may be a feasible alternative to additional mock OSCEs to support student preparation.

Comparison of student pharmacists' performance on in-person vs. virtual OSCEs in a pre-APPE capstone course

INTRODUCTION

Objective structured clinical examinations (OSCEs,) provided in-person or virtually, assess student pharmacist readiness for advanced pharmacy practice experiences (APPEs). During the COVID-19 pandemic in 2020, it was necessary for many educators to design and implement virtual OSCEs (vOSCEs). Impact on student performance utilizing in-person vs. vOSCE has not been well evaluated. The objective of this study was to determine if a difference existed in student performance when comparing in-person vs. vOSCE in a third year (P3) pharmacy pre-APPE capstone course.

METHODS

In winter 2019, four in-person OSCE stations were designed and implemented in a pre-APPE P3 capstone course. In winter 2021, the same four stations were transitioned into vOSCE stations. Assessment (summative) data from similar student cohorts from OSCE 2019 were compared vOSCE 2021 stations using Mann-Whitney U test.

RESULTS

There was no meaningful difference found when comparing student performance on in-person OSCE vs. vOSCE. There was no significant difference for the number of students offered remediation. For most stations, performance improved from formative to summative assessments.

CONCLUSIONS

Providing vOSCEs to assess students' skills in a pre-APPE P3 capstone course is a reasonable alternative to in-person OSCEs.

Adaptations in clinical examinations of medical students in response to the COVID-19 pandemic: a systematic review

INTRODUCTION

Clinical examinations (assessments) are integral to ensuring that medical students can treat patients safely and effectively. The COVID-19 pandemic disrupted traditional formats of clinical examinations. This prompted Medical Schools to adapt their approaches to conducting these examinations to make them suitable for delivery in the pandemic. This systematic review aims to identify the approaches that Medical Schools, internationally, adopted in adapting their clinical examinations of medical students in response to the COVID-19 pandemic.

METHODS

Three databases and four key medical education journals were systematically searched up to 22 October 2021; a grey literature search was also undertaken. Two reviewers independently screened at title, abstract stage and full text stage against predefined eligibility criteria. Discrepancies were resolved by discussion and involvement of senior authors. Risk of bias assessment was performed using an adapted version of a pre-existing risk of bias assessment tool for medical education developments. Results were summarised in a narrative synthesis.

RESULTS

A total of 36 studies were included, which documented the approaches of 48 Medical Schools in 17 countries. Approaches were categorised into in-person clinical examinations (22 studies) or online clinical examinations (14 studies). Authors of studies reporting in-person clinical examinations described deploying enhanced infection control measures along with modified patient participation. Authors of studies reporting online clinical examinations described using online software to create online examination circuits. All authors reported that adapted examinations were feasible, scores were comparable to previous years' student cohorts, and participant feedback was positive. Risk of bias assessment highlighted heterogeneity in reporting of the clinical examinations.

CONCLUSIONS

This review identified two broad approaches to adapting clinical examinations in the pandemic: in-person and online. Authors reported it was feasible to conduct clinical examinations in the pandemic where medical educators are given sufficient time and resources to carefully plan and introduce suitable adaptations. However, the risk of bias assessment identified few studies with high reporting quality, which highlights the need for a common framework for reporting of medical education developments to enhance reproducibility across wider contexts. Our review provides medical educators with the opportunity to reflect on past practises and facilitate the design and planning of future examinations.

Students’ and Examiners’ Experiences of Their First Virtual Pharmacy Objective Structured Clinical Examination (OSCE) in Australia during the COVID-19 Pandemic

Objective Structured Clinical Examinations (OSCEs) are routinely used in healthcare education programs. Traditionally, students undertake OSCEs as face-to-face interactions to assess competency in soft skills. Due to physical distancing restrictions during COVID-19, alternative methods were required. This study utilized a mixed-method design (online survey and interviews) to evaluate second-year pharmacy students’ and examiners’ experiences of their first virtual OSCEs in Australia. A total of 196 students completed their first virtual OSCE in June 2020 of which 190 students completed the online survey. However, out of the 190 students, only 88% (n = 167) consented to the use of the data from their online survey. A further 10 students and 12 examiners were interviewed. Fifty-five students (33%) who participated in the online survey strongly agreed or agreed that they preferred the virtual experience to face-to-face OSCEs while 44% (n = 73) neither agreed nor disagreed. Only 20% (n = 33) felt more anxious with the virtual OSCEs. Additionally, thematic analysis found non-verbal communication as a barrier during the OSCE. Positive aspects about virtual OSCEs included flexibility, decreased levels of anxiety and relevance with emerging telehealth practice. The need for remote online delivery of assessments saw innovative ways of undertaking OSCEs and an opportunity to mimic telehealth. While students and examiners embraced the virtual OSCE process, face-to-face OSCEs were still considered important and irreplaceable. Future opportunities for OSCEs to be delivered both face-to-face and virtually should be considered.

Evaluation of the Utility of Online Objective Structured Clinical Examination Conducted During the COVID-19 Pandemic

BACKGROUND

The COVID-19 pandemic led to profound restrictions on the face-to-face learning and assessment in all educational institutions, particularly the medical schools. The College of Medicine and Medical Sciences of the Arabian Gulf University (CMMS-AGU) conducted the final exams, both theoretical and clinical components, for its MD students online. This study was conducted to evaluate the utility of online clinical exams held at CMMS-AGU.

METHODS

This is a cross-sectional, mixed method study that included samples from final year medical students, examiners, and heads of clinical departments. Data were collected through surveys, structured interviews, documents' review, and calculation of online examination's psychometrics. Descriptive statistics were used. Quantitative data were presented in the form of means and standard deviations. Responses of heads of clinical departments in the structured interview were transcribed and analyzed thematically based on three pre-established themes.

RESULTS

Quantitative and qualitative data on the utility (validity, reliability, acceptability, educational impact, and cost and feasibility) of online objective structured clinical examination (OSCE) were collected. Content validity of the online clinical examination was established through high mean scores of content representativeness, which was confirmed by the heads of clinical departments regarding the proper coverage of clinical skills. Criterion validity was established through a high correlation between clinical and theoretical exam results (r = 0.75). Reliability of the exam was established through an acceptable Cronbach's alpha value (0.70 to 0.78) over the four days of the examinations. The examinations were perceived as highly acceptable by both students and examiners. High educational impact was inferred from students' responses and review of documents. The examination was found to be feasible and of reasonable cost.

CONCLUSION

Online OSCE might be a good alternative of conventional clinical assessments in times of crises and impossibility of having in-person contact between students, examiners, and patients. An important major drawback is still present in such initiatives, which is the inability to assess students' physical examination skills.