1
|
Hara S, Ohta K, Aono D, Tamai T, Kurachi M, Sugimori K, Mihara H, Ichimura H, Yamamoto Y, Nomura H. Feasibility and reliability of the pandemic-adapted online-onsite hybrid graduation OSCE in Japan. ADVANCES IN HEALTH SCIENCES EDUCATION : THEORY AND PRACTICE 2024; 29:949-965. [PMID: 37851159 PMCID: PMC11208196 DOI: 10.1007/s10459-023-10290-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/09/2023] [Accepted: 09/24/2023] [Indexed: 10/19/2023]
Abstract
Objective structured clinical examination (OSCE) is widely used to assess medical students' clinical skills. Virtual OSCEs were used in place of in-person OSCEs during the COVID-19 pandemic; however, their reliability is yet to be robustly analyzed. By applying generalizability (G) theory, this study aimed to evaluate the reliability of a hybrid OSCE, which admixed in-person and online methods, and gain insights into improving OSCEs' reliability. During the 2020-2021 hybrid OSCEs, one examinee, one rater, and a vinyl mannequin for physical examination participated onsite, and a standardized simulated patient (SP) for medical interviewing and another rater joined online in one virtual breakout room on an audiovisual conferencing system. G-coefficients and 95% confidence intervals of the borderline score, namely border zone (BZ), under the standard 6-station, 2-rater, and 6-item setting were calculated. G-coefficients of in-person (2017-2019) and hybrid OSCEs (2020-2021) under the standard setting were estimated to be 0.624, 0.770, 0.782, 0.759, and 0.823, respectively. The BZ scores were estimated to be 2.43-3.57, 2.55-3.45, 2.59-3.41, 2.59-3.41, and 2.51-3.49, respectively, in the score range from 1 to 6. Although hybrid OSCEs showed reliability comparable to in-person OSCEs, they need further improvement as a very high-stakes examination. In addition to increasing clinical vignettes, having more proficient online/on-demand raters and/or online SPs for medical interviews could improve the reliability of OSCEs. Reliability can also be ensured through supplementary examination and by increasing the number of online raters for a small number of students within the BZs.
Collapse
Affiliation(s)
- Satoshi Hara
- Medical Education Research Center, Kanazawa University, Kanazawa, Japan
| | - Kunio Ohta
- Medical Education Research Center, Kanazawa University, Kanazawa, Japan
| | - Daisuke Aono
- Medical Education Research Center, Kanazawa University, Kanazawa, Japan
| | - Toshikatsu Tamai
- Medical Education Research Center, Kanazawa University, Kanazawa, Japan
- Department of Molecular Genetics, Kanazawa University, Kanazawa, Japan
| | - Makoto Kurachi
- Medical Education Research Center, Kanazawa University, Kanazawa, Japan
- Department of Molecular Genetics, Kanazawa University, Kanazawa, Japan
| | - Kimikazu Sugimori
- Center for the Advancement of Higher Education, Hokuriku University, Kanazawa, Japan
| | - Hiroshi Mihara
- Center for Medical Education and Career Development, Toyama University, Toyama, Japan
| | - Hiroshi Ichimura
- Medical Education Research Center, Kanazawa University, Kanazawa, Japan
- Department of Viral Infection and International Health, Kanazawa University, Kanazawa, Japan
| | - Yasuhiko Yamamoto
- Medical Education Research Center, Kanazawa University, Kanazawa, Japan
- Department of Biochemistry and Molecular Vascular Biology, Faculty of Medicine, Institute of Medical, Pharmaceutical, and Health Sciences, Kanazawa University, Kanazawa, Japan
| | - Hideki Nomura
- Medical Education Research Center, Kanazawa University, Kanazawa, Japan.
- Department of General Medicine, Kanazawa University Hospital, 13-1 Takara-machi, Kanazawa, Ishikawa, 920-8641, Japan.
| |
Collapse
|
2
|
Reay RE, Maguire PA, Looi JC. A level playing field? Evaluation of the virtual Objective Structured Clinical Examination in Psychiatry and Addiction Medicine: A mixed methods study. Australas Psychiatry 2024:10398562241249567. [PMID: 38739630 DOI: 10.1177/10398562241249567] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 05/16/2024]
Abstract
OBJECTIVE The Objective Structured Clinical Examination (OSCE) has been used for clinical assessment of a broad range of medical student competencies in Psychiatry and Addiction Medicine. However, there has been little research into online assessments. We investigated the virtual OSCE (v-OSCE) from the user perspective to better understand its acceptability, usefulness, benefits, challenges and potential improvements. METHODS At the conclusion of the v-OSCE, all participants (medical students, examiners and simulated patients) were invited to participate in a brief online survey, based on the Technology and Acceptance Model. Freeform qualitative feedback was also obtained to explore participants' experiences and attitudes. RESULTS Participants reported the v-OSCE was acceptable, efficient, convenient and easy to use. It was perceived as useful for demonstrating students' interviewing skills and interacting with the simulated patient. Benefits included greater convenience, reduced stress and travel time. Challenges were similar to those experienced in 'real world' telepsychiatry, primarily related to assessment of non-verbal cues and emotional prosody. CONCLUSIONS Our findings inform recommendations for improving online examinations. These include increased preparation, practice and professionalism, to better simulate the in-person experience. Study credibility was strengthened by the triangulation of qualitative, quantitative and psychometric data.
Collapse
Affiliation(s)
- Rebecca E Reay
- Academic Unit of Psychiatry and Addiction Medicine, Canberra Hospital, Australian National University School of Medicine and Psychology, Garran, ACT, Australia
| | - Paul A Maguire
- Academic Unit of Psychiatry and Addiction Medicine, Canberra Hospital, Australian National University School of Medicine and Psychology, Canberra, ACT, Australia
| | - Jeffrey Cl Looi
- Academic Unit of Psychiatry and Addiction Medicine, Canberra Hospital, Australian National University School of Medicine and Psychology, Canberra, ACT, Australia; and Consortium of Australian-Academic Psychiatrists for Independent Policy and Research Analysis (CAPIPRA), Canberra, ACT, Australia
| |
Collapse
|
3
|
Wolfe RM, Hant FN, Ishizawar RC, Criscione-Schreiber LG, Jonas BL, O'Rourke KS, Bolster MB. Virtual Learning and Assessment in Rheumatology Fellowship Training: Objective Structured Clinical Examination Revisited. Arthritis Care Res (Hoboken) 2023; 75:2435-2441. [PMID: 37291752 DOI: 10.1002/acr.25170] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2022] [Revised: 04/27/2023] [Accepted: 06/06/2023] [Indexed: 06/10/2023]
Abstract
OBJECTIVE With the onset of the COVID-19 pandemic, an annual multi-institutional face-to-face rheumatology objective structured clinical examination (ROSCE) was transformed into a virtual format. The educational goals of the virtual ROSCE (vROSCE) were to reproduce the educational value of the previous in-person ROSCE, providing a valuable formative assessment of rheumatology training activities encompassing the 6 Accreditation Council for Graduate Medical Education (ACGME) core competencies for fellows-in-training (FITs). This article describes the novel design, feasibility, and stakeholder value of a vROSCE. METHODS Through an established collaboration of 5 rheumatology fellowship training programs, in February 2021, a vROSCE was created and conducted using a Zoom platform. Station development included learning objectives, FIT instructions, faculty proctor instructions, and a checklist by which to provide structured formative feedback. An anonymous, optional web-based survey was sent to FIT participants to evaluate the experience. RESULTS Twenty-three rheumatology FITs from 5 institutions successfully rotated through 6 stations in the vROSCE. Immediate feedback was given to each FIT using standardized rubrics structured around ACGME core competencies. A total of 65% of FITs (15 of 23) responded to the survey, and 93% of survey respondents agreed or strongly agreed that the vROSCE was a helpful educational activity and identified individualized opportunities for improvement. CONCLUSION A vROSCE is an innovative, feasible, valuable, and well-received educational technology tool. The vROSCE enriched rheumatology FITs' education and offered collaborative learning experiences across institutions.
Collapse
Affiliation(s)
- Rachel M Wolfe
- Wake Forest School of Medicine, Winston-Salem, North Carolina
| | - Faye N Hant
- Medical University of South Carolina, Charleston
| | | | | | - Beth L Jonas
- University of North Carolina at Chapel Hill, Chapel Hill
| | | | - Marcy B Bolster
- Harvard Medical School and Massachusetts General Hospital, Boston
| |
Collapse
|
4
|
Giri J, Stewart C. Innovations in assessment in health professions education during the COVID-19 pandemic: A scoping review. CLINICAL TEACHER 2023; 20:e13634. [PMID: 37698032 DOI: 10.1111/tct.13634] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/29/2023] [Accepted: 07/27/2023] [Indexed: 09/13/2023]
Abstract
BACKGROUND After an initial period of uncertainty during the COVID-19 pandemic, medical universities responded to the pandemic by innovating their methods of teaching-learning and assessment. This scoping review aimed to identify innovations in assessment in medical, dental and nursing education at the undergraduate and postgraduate levels during the pandemic. METHODS This review was conducted utilising Arksey and O'Malley's framework where three electronic databases-PubMed, Embase and Web of Science-were systematically searched in February 2022 for articles in English describing innovations in assessments published in or after January 2020. RESULTS A total of 70 articles were included in the review. Among the primary research articles, 82.1% reported online assessment, 76.1% were related to medical speciality, 74.6% described assessment at the undergraduate level, 68.7% described summative assessment and 58.2% reported assessment of practical skills. All assessments of theoretical knowledge were done virtually with MCQ being the most used assessment tool and virtual OSCE being the most popular assessment tool to assess practical skills; however, it was acknowledged that the assessment of procedural skills was its limitation. Although the studies described the newer forms of assessments as feasible and acceptable, few studies reported the reliability and validity of the assessments. CONCLUSIONS This scoping review identified several innovations in assessment during the COVID-19 pandemic with an overall shift towards virtual or hybrid forms of assessment. However, there was a dearth of evidence regarding the effectiveness of these assessment tools suggesting a need for more research before the tools can be implemented post-pandemic.
Collapse
Affiliation(s)
- Jamal Giri
- Department of Orthodontics, B.P. Koirala Institute of Health Sciences, Dharan, Nepal
| | - Claire Stewart
- Medical Education, University of Nottingham, Nottingham, UK
| |
Collapse
|
5
|
The Challenges of Visualisation During Delivery and Assessment of Clinical Teaching Under COVID-19: A Reflective Account of Problem Solving. ADVANCES IN EXPERIMENTAL MEDICINE AND BIOLOGY 2023; 1397:199-213. [DOI: 10.1007/978-3-031-17135-2_11] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/23/2022]
|
6
|
Motkur V, Bharadwaj A, Yogarajah N. Is online objective structured clinical examination teaching an acceptable replacement in post-COVID-19 medical education in the United Kingdom?: a descriptive study. JOURNAL OF EDUCATIONAL EVALUATION FOR HEALTH PROFESSIONS 2022; 19:30. [PMID: 36344197 PMCID: PMC9807458 DOI: 10.3352/jeehp.2022.19.30] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/09/2022] [Accepted: 10/17/2022] [Indexed: 06/16/2023]
Abstract
PURPOSE Coronavirus disease 2019 (COVID-19) restrictions resulted in an increased emphasis on virtual communication in medical education. This study assessed the acceptability of virtual teaching in an online objective structured clinical examination (OSCE) series and its role in future education. METHODS Six surgical OSCE stations were designed, covering common surgical topics, with specific tasks testing data interpretation, clinical knowledge, and communication skills. These were delivered via Zoom to students who participated in student/patient/examiner role-play. Feedback was collected by asking students to compare online teaching with previous experiences of in-person teaching. Descriptive statistics were used for Likert response data, and thematic analysis for free-text items. RESULTS Sixty-two students provided feedback, with 81% of respondents finding online instructions preferable to paper equivalents. Furthermore, 65% and 68% found online teaching more efficient and accessible, respectively, than in-person teaching. Only 34% found communication with each other easier online; Forty percent preferred online OSCE teaching to in-person teaching. Students also expressed feedback in positive and negative free-text comments. CONCLUSION The data suggested that generally students were unwilling for online teaching to completely replace in-person teaching. The success of online teaching was dependent on the clinical skill being addressed; some were less amenable to a virtual setting. However, online OSCE teaching could play a role alongside in-person teaching.
Collapse
Affiliation(s)
- Vashist Motkur
- East and North Hertfordshire NHS Trust, Stevenage, UK
- University College London Medical School, London, UK
| | - Aniket Bharadwaj
- University College London Medical School, London, UK
- Cambridge University Hospitals NHS Foundation Trust, Cambridge, UK
| | - Nimalesh Yogarajah
- University College London Medical School, London, UK
- West Hertfordshire Hospitals NHS Trust, Watford, UK
| |
Collapse
|
7
|
Yang YY, Huang PH, Yang LY, Huang CC, Liu CW, Huang SS, Chen CH, Lee FY, Kao SY, Shulruf B. Equal Z standard-setting method to estimate the minimum number of panelists for a medical school’s objective structured clinical examination in Taiwan: a simulation study. JOURNAL OF EDUCATIONAL EVALUATION FOR HEALTH PROFESSIONS 2022; 19:27. [PMID: 36252990 PMCID: PMC9764018 DOI: 10.3352/jeehp.2022.19.27] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 06/22/2022] [Accepted: 09/08/2022] [Indexed: 06/16/2023]
Abstract
PURPOSE Undertaking a standard-setting exercise is a common method for setting pass/fail cut scores for high-stakes examinations. The recently introduced equal Z standard-setting method (EZ method) has been found to be a valid and effective alternative for the commonly used Angoff and Hofstee methods and their variants. The current study aims to estimate the minimum number of panelists required for obtaining acceptable and reliable cut scores using the EZ method. METHODS The primary data were extracted from 31 panelists who used the EZ method for setting cut scores for a 12-station of medical school’s final objective structured clinical examination (OSCE) in Taiwan. For this study, a new data set composed of 1,000 random samples of different panel sizes, ranging from 5 to 25 panelists, was established and analyzed. Analysis of variance was performed to measure the differences in the cut scores set by the sampled groups, across all sizes within each station. RESULTS On average, a panel of 10 experts or more yielded cut scores with confidence more than or equal to 90% and 15 experts yielded cut scores with confidence more than or equal to 95%. No significant differences in cut scores associated with panel size were identified for panels of 5 or more experts. CONCLUSION The EZ method was found to be valid and feasible. Less than an hour was required for 12 panelists to assess 12 OSCE stations. Calculating the cut scores required only basic statistical skills.
Collapse
Affiliation(s)
- Ying-Ying Yang
- Division of Clinical Skills Training Center, Taipei Veterans General Hospital, Taipei, Taiwan
- National Yang-Ming University, Taipei, Taiwan
| | - Pin-Hsiang Huang
- National Yang-Ming University, Taipei, Taiwan
- University of New South Wales, Sydney, Australia
- Department of Medical Education, Taipei Veterans General Hospital, Taipei, Taiwan
| | - Ling-Yu Yang
- Department of Medical Education, Taipei Veterans General Hospital, Taipei, Taiwan
| | - Chia-Chang Huang
- Division of Clinical Skills Training Center, Taipei Veterans General Hospital, Taipei, Taiwan
- National Yang-Ming University, Taipei, Taiwan
| | - Chih-Wei Liu
- Division of Clinical Skills Training Center, Taipei Veterans General Hospital, Taipei, Taiwan
- National Yang-Ming University, Taipei, Taiwan
| | - Shiau-Shian Huang
- Division of Clinical Skills Training Center, Taipei Veterans General Hospital, Taipei, Taiwan
- National Yang-Ming University, Taipei, Taiwan
| | - Chen-Huan Chen
- National Yang-Ming University, Taipei, Taiwan
- Department of Medical Education, Taipei Veterans General Hospital, Taipei, Taiwan
| | - Fa-Yauh Lee
- Department of Medical Education, Taipei Veterans General Hospital, Taipei, Taiwan
| | - Shou-Yen Kao
- Department of Medical Education, Taipei Veterans General Hospital, Taipei, Taiwan
| | - Boaz Shulruf
- University of New South Wales, Sydney, Australia
- University of Auckland, Auckland, New Zealand
| |
Collapse
|
8
|
Gortney JS, Fava JP, Berti AD, Stewart B. Comparison of student pharmacists' performance on in-person vs. virtual OSCEs in a pre-APPE capstone course. CURRENTS IN PHARMACY TEACHING & LEARNING 2022; 14:1116-1121. [PMID: 36154957 PMCID: PMC9352434 DOI: 10.1016/j.cptl.2022.07.026] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/01/2021] [Revised: 07/14/2022] [Accepted: 07/20/2022] [Indexed: 06/16/2023]
Abstract
INTRODUCTION Objective structured clinical examinations (OSCEs,) provided in-person or virtually, assess student pharmacist readiness for advanced pharmacy practice experiences (APPEs). During the COVID-19 pandemic in 2020, it was necessary for many educators to design and implement virtual OSCEs (vOSCEs). Impact on student performance utilizing in-person vs. vOSCE has not been well evaluated. The objective of this study was to determine if a difference existed in student performance when comparing in-person vs. vOSCE in a third year (P3) pharmacy pre-APPE capstone course. METHODS In winter 2019, four in-person OSCE stations were designed and implemented in a pre-APPE P3 capstone course. In winter 2021, the same four stations were transitioned into vOSCE stations. Assessment (summative) data from similar student cohorts from OSCE 2019 were compared vOSCE 2021 stations using Mann-Whitney U test. RESULTS There was no meaningful difference found when comparing student performance on in-person OSCE vs. vOSCE. There was no significant difference for the number of students offered remediation. For most stations, performance improved from formative to summative assessments. CONCLUSIONS Providing vOSCEs to assess students' skills in a pre-APPE P3 capstone course is a reasonable alternative to in-person OSCEs.
Collapse
Affiliation(s)
- Justine S Gortney
- Wayne State University, Department of Pharmacy Practice, Eugene Applebaum College of Pharmacy and Health Sciences, 259 Mack Avenue, Detroit, MI 48201, United States.
| | - Joseph P Fava
- Wayne State University, Department of Pharmacy Practice, Eugene Applebaum College of Pharmacy and Health Sciences, 259 Mack Avenue, Detroit, MI 48201, United States.
| | - Andrew D Berti
- Wayne State University, Department of Pharmacy Practice, Eugene Applebaum College of Pharmacy and Health Sciences, 259 Mack Avenue, Detroit, MI 48201, United States.
| | - Brittany Stewart
- Wayne State University, Department of Pharmacy Practice, Eugene Applebaum College of Pharmacy and Health Sciences, 259 Mack Avenue, Detroit, MI 48201, United States.
| |
Collapse
|
9
|
Arekat M, Shehata MH, Deifalla A, Al-Ansari A, Kumar A, Alsenbesy M, Alshenawi H, El-Agroudy A, Husni M, Rizk D, Elamin A, Ben Salah A, Atwa H. Evaluation of the Utility of Online Objective Structured Clinical Examination Conducted During the COVID-19 Pandemic. ADVANCES IN MEDICAL EDUCATION AND PRACTICE 2022; 13:407-418. [PMID: 35509352 PMCID: PMC9060808 DOI: 10.2147/amep.s357229] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/19/2022] [Accepted: 03/28/2022] [Indexed: 05/05/2023]
Abstract
BACKGROUND The COVID-19 pandemic led to profound restrictions on the face-to-face learning and assessment in all educational institutions, particularly the medical schools. The College of Medicine and Medical Sciences of the Arabian Gulf University (CMMS-AGU) conducted the final exams, both theoretical and clinical components, for its MD students online. This study was conducted to evaluate the utility of online clinical exams held at CMMS-AGU. METHODS This is a cross-sectional, mixed method study that included samples from final year medical students, examiners, and heads of clinical departments. Data were collected through surveys, structured interviews, documents' review, and calculation of online examination's psychometrics. Descriptive statistics were used. Quantitative data were presented in the form of means and standard deviations. Responses of heads of clinical departments in the structured interview were transcribed and analyzed thematically based on three pre-established themes. RESULTS Quantitative and qualitative data on the utility (validity, reliability, acceptability, educational impact, and cost and feasibility) of online objective structured clinical examination (OSCE) were collected. Content validity of the online clinical examination was established through high mean scores of content representativeness, which was confirmed by the heads of clinical departments regarding the proper coverage of clinical skills. Criterion validity was established through a high correlation between clinical and theoretical exam results (r = 0.75). Reliability of the exam was established through an acceptable Cronbach's alpha value (0.70 to 0.78) over the four days of the examinations. The examinations were perceived as highly acceptable by both students and examiners. High educational impact was inferred from students' responses and review of documents. The examination was found to be feasible and of reasonable cost. CONCLUSION Online OSCE might be a good alternative of conventional clinical assessments in times of crises and impossibility of having in-person contact between students, examiners, and patients. An important major drawback is still present in such initiatives, which is the inability to assess students' physical examination skills.
Collapse
Affiliation(s)
- Mona Arekat
- Internal Medicine Department, College of Medicine and Medical Sciences, Arabian Gulf University, Manama, Kingdom of Bahrain
| | - Mohamed Hany Shehata
- Family and Community Medicine Department, College of Medicine and Medical Sciences, Arabian Gulf University, Manama, Kingdom of Bahrain
- Family Medicine Department, Faculty of Medicine, Helwan University, Cairo, Egypt
- Correspondence: Mohamed Hany Shehata, Family and Community Medicine Department, College of Medicine and Medical Sciences, Arabian Gulf University, Manama, Kingdom of Bahrain, Tel +97333918183, Email
| | - Abdelhalim Deifalla
- Anatomy Department, College of Medicine and Medical Sciences, Arabian Gulf University, Manama, Bahrain
- Human Anatomy and Embryology Department, Faculty of Medicine, Suez Canal University, Ismailia, Egypt
| | - Ahmed Al-Ansari
- Medical Education Unit, College of Medicine and Medical Sciences, Arabian Gulf University, Manama, Bahrain
| | - Archana Kumar
- Medical Education Unit, College of Medicine and Medical Sciences, Arabian Gulf University, Manama, Bahrain
| | - Mohamed Alsenbesy
- Internal Medicine Department, College of Medicine and Medical Sciences, Arabian Gulf University, Manama, Kingdom of Bahrain
- Internal Medicine Department, Faculty of Medicine, South Valley University, Qena, Egypt
| | - Hamdi Alshenawi
- Surgery Department, College of Medicine and Medical Sciences, Arabian Gulf University, Manama, Bahrain
| | - Amgad El-Agroudy
- Internal Medicine Department, College of Medicine and Medical Sciences, Arabian Gulf University, Manama, Kingdom of Bahrain
| | - Mariwan Husni
- Psychiatry Department, College of Medicine and Medical Sciences, Arabian Gulf University, Manama, Bahrain
- Psychiatry Department, Northern Ontario School of Medicine (NOSM), Ontario, Canada
| | - Diaa Rizk
- Obstetrics and Gynecology Department, College of Medicine and Medical Sciences, Arabian Gulf University, Manama, Bahrain
| | - Abdelaziz Elamin
- Pediatrics Department, College of Medicine and Medical Sciences, Arabian Gulf University, Manama, Bahrain
| | - Afif Ben Salah
- Family and Community Medicine Department, College of Medicine and Medical Sciences, Arabian Gulf University, Manama, Kingdom of Bahrain
| | - Hani Atwa
- Medical Education Unit, College of Medicine and Medical Sciences, Arabian Gulf University, Manama, Bahrain
- Medical Education Department, Faculty of Medicine, Suez Canal University, Ismailia, Egypt
| |
Collapse
|
10
|
Hung CT, Fang SA, Liu FC, Hsu CH, Yu TY, Wang WM. Applying the Student Response System in the Online Dermatologic Video Curriculum on Medical Students' Interaction and Learning Outcomes during the COVID-19 Pandemic. Indian J Dermatol 2022; 67:477. [PMID: 36578722 PMCID: PMC9792048 DOI: 10.4103/ijd.ijd_147_22] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/30/2022] Open
Abstract
Background The coronavirus disease 2019 (COVID-19) pandemic impacted medical education worldwide. Online lecture is increasingly prevalent in higher education, but students' completion rate is quite low. Aims This study aimed to determine the effectiveness of the student response system (SRS) in the online dermatologic video curriculum on medical students. Methods A prospective study was conducted on 176 undergraduate fourth-year medical students. The online video lecture was integrated with SRS. Results A total of 173 students completed the pre-test, and the attendance rate (pre-test/total) was 98.3%. A total of 142 students completed the post-test, and the completion rate (post-test/pre-test) was 82.8%. The post-test score (83.69 ± 4.34) was found to be significantly higher than that of the pre-test (62.69 ± 6.08, P =0.0002). A total of 138 students completed the questionnaire, and 92% of students opined that SRS was easy to operate. 86% of students agreed with the fact that the use of SRS could increase their learning performance by interacting with teachers. In the open-ended question, students stated that SRS offered opportunities for student-faculty interaction, allowed them to get immediate feedback, and promote active participation. Conclusions These results highlight that the integration of SRS in the online video curriculum increases students' completion rates and learning outcomes. Moreover, the SRS is easy to operate for the students and enhances student-faculty interaction. The SRS may be adopted in online learning during this challenging time.
Collapse
Affiliation(s)
- Chih-Tsung Hung
- From the Department of Dermatology, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan,Graduate Institute of Medical Sciences, School of Medicine, National Defense Medical Center, Taipei, Taiwan,Department of General Medicine, Tri-Service General Hospital, Taipei, Taiwan
| | - Shao-An Fang
- Department of General Medicine, Tri-Service General Hospital, Taipei, Taiwan
| | - Feng-Cheng Liu
- Department of Rheumatology/ Immunology and Allergy, Department of Internal Medicine, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan
| | - Chih-Hsiung Hsu
- School of Public Health, School of Medicine, National Defense Medical Center, Taipei, Taiwan
| | - Ting-Yu Yu
- Department of General Medicine, Tri-Service General Hospital, Taipei, Taiwan
| | - Wei-Ming Wang
- From the Department of Dermatology, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan,Graduate Institute of Medical Sciences, School of Medicine, National Defense Medical Center, Taipei, Taiwan,Vice Chairman, School of Medicine, National Defense Medical Center, Taipei, Taiwan,Address for correspondence: Dr. Wei-Ming Wang, Department of Dermatology, Tri-Service General Hospital, National Defense Medical Center, No. 325, Sec. 2, Chenggong Rd., Neihu Dist., Taipei City 114, Taiwan (R.O.C.). E-mail:
| |
Collapse
|
11
|
Chu K, Sathanandan S. The virtual Clinical Assessment of Skills and Competence: the impact and challenges of a digitised final examination. BJPsych Bull 2021; 47:110-115. [PMID: 34937596 PMCID: PMC10063989 DOI: 10.1192/bjb.2021.112] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/23/2022] Open
Abstract
The COVID-19 pandemic has affected how clinical examinations are conducted, resulting in the Royal College of Psychiatrists delivering the Clinical Assessment of Skills and Competence virtually. Although this pragmatic step has allowed for progression of training, it has come at the cost of a significantly altered examination experience. This article aims to explore the fairness of such an examination, the difference in trainee experience, and the use of telemedicine to consider what might be lost as well as gained at a time when medical education and delivery of healthcare are moving toward the digitised frontier.
Collapse
Affiliation(s)
- Kenny Chu
- St Pancras Hospital, Camden and Islington NHS Foundation Trust, UK
| | | |
Collapse
|
12
|
Felthun JZ, Taylor S, Shulruf B, Allen DW. Empirical analysis comparing the tele-objective structured clinical examination (teleOSCE) and the in-person assessment in Australia. JOURNAL OF EDUCATIONAL EVALUATION FOR HEALTH PROFESSIONS 2021; 18:23. [PMID: 34551510 PMCID: PMC8616724 DOI: 10.3352/jeehp.2021.18.23] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/19/2021] [Accepted: 09/20/2021] [Indexed: 06/13/2023]
Abstract
PURPOSE It aimed to compare the use of the tele objective structured clinical examination (teleOSCE) with in-person assessment in high-stakes clinical examination so as to determine the impact of the teleOSCE on the assessment undertaken. Discussion follows regarding what skills and domains can effectively be assessed in a teleOSCE. METHODS This study is a retrospective observational analysis. It compares the results achieved by final year medical students in their clinical examination, assessed using the teleOSCE in 2020 (n=285), with those who were examined using the traditional in-person format in 2019 (n=280). The study was undertaken at the University of New South Wales, Australia. RESULTS In the domain of physical examination, students in 2020 scored 0.277 points higher than those in 2019 (mean difference -0.277, P<0.001, effect size 0.332). Across all other domains, there was no significant difference in mean scores between 2019 and 2020. CONCLUSION The teleOSCE does not negatively impact assessment in clinical examination in all domains except physical examination. If the teleOSCE is the future of clinical skills examination, assessment of physical examination will require concomitant workplace-based assessment.
Collapse
Affiliation(s)
| | - Silas Taylor
- Office of Medical Education, University of New South Wales, Sydney, Australia
| | - Boaz Shulruf
- Office of Medical Education, University of New South Wales, Sydney, Australia
- Centre for Medical and Health Sciences Education, Faculty of Medical and Health Sciences, University of Auckland, Auckland, New Zealand
| | - Digby Wigram Allen
- School of Medicine, The University of New South Wales, Kensington, Australia
| |
Collapse
|