Resident education during Covid-19, virtual mock OSCE's via zoom: A pilot program

Digital vs Conventional OSCE in Orthopedic Surgery: A Feasibility Cross-Sectional Study

OBJECTIVE

Remote OSCEs (Objective Structured Clinical Examination) are an alternative evaluation method during pandemic periods but they have never been evaluated in orthopedic surgery. We aimed to evaluate whether remote OSCEs would be feasible, and efficient for assessment of undergraduate medical students.

METHODS

A cross-sectional study was performed. Thirty-four students were randomly assigned into 2 equal groups, either the conventional OSCE group or the digital OSCE group. Three types of skills were assessed: technical procedure, clinical examination, and radiographic analysis. Students were graded and they filled in a satisfaction questionnaire for both types of OSCEs.

RESULTS

The mean score, out of 20, was 14.3 ± 2.5 (range 9.3-19) for the digital sessions, versus 14.4 ± 2.3 (range 10-18.6) for conventional sessions (p = 0.81). Bland Altman Plot showed that 88% of students scored within agreement. The average global feedback was different for item repeatability, relevance, and OSCEs preference (p < 0.0001, p = 0.0001, and p < 0.0001 respectively). However, they did not report differences for the item concerning the organization (p = 0.2).

CONCLUSION

The results of this comparative study between digital and conventional OSCEs showed comparable distance learning scores between the 2 groups, whatever the skill assessed. However, the student's evaluation showed some reticence to conduct again OSCEs remotely.

A comparison of the American Board of Anesthesiology's in-person and virtual objective structured clinical examinations

BACKGROUND

The American Board of Anesthesiology's Objective Structured Clinical Examination (OSCE), as a component of its initial certification process, had been administered in-person in a dedicated assessment center since its launch in 2018 until March 2020. Due to the COVID-19 pandemic, a virtual format of the exam was piloted in December 2020 and was administered in 2021. This study aimed to compare candidate performance, examiner grading severity, and scenario difficulty between these two formats of the OSCE.

METHODS

The Many-Facet Rasch Model was utilized to estimate candidate performance, examiner grading severity, and scenario difficulty for the in-person and virtual OSCEs separately. The virtual OSCE was equated to the in-person OSCE by common examiners and common scenarios. Independent-samples t-test was used to compare candidate performance, and partially overlapping samples t-tests were applied to compare examiner grading severity and scenario difficulty between the in-person and virtual OSCEs.

RESULTS

The in-person (n = 3235) and virtual (n = 2934) first-time candidates were comparable in age, sex, race/ethnicity, and whether U.S. medical school graduates. The virtual scenarios (n = 35, mean [0.21] ± SD [0.38] in logits) were more difficult than the in-person scenarios (n = 93, 0.00 ± 0.69, Welch's partially overlapping samples t-test, p = 0.01); there were no statistically significant differences in examiner severity (n = 390, -0.01 ± 0.82 vs. n = 304, -0.02 ± 0.93, Welch's partially overlapping samples t-test, p = 0.81) or candidate performance (2.19 ± 0.93 vs. 2.18 ± 0.92, Welch's independent samples t-test, p = 0.83) between the in-person and virtual OSCEs.

CONCLUSIONS

Our retrospective analyses of first-time OSCEs found comparable candidate performance and examiner grading severity between the in-person and virtual formats, despite the virtual scenarios being more difficult than the in-person scenarios. These results provided assurance that the virtual OSCE functioned reasonably well in a high-stakes setting.

Birth of a Social Mediatrician: Adopting Slack, Twitter, and Instagram for Residents

Introduction Physicians have increasingly used social media platforms to review new research, expand networks, and communicate. However, few studies have evaluated how the integration of social media into residency programs affects training. This is relevant during the COVID-19 pandemic, with a shift towards virtual formats for medical education, community building, and recruitment. Objective The objective of this study was to evaluate how the integration of social media platforms, including Slack, Twitter, and Instagram, influences education, social connectedness, and recruitment within a residency program. Methods In 2020, pediatric residents at one institution were encouraged to create personal Twitter and Instagram accounts if they did not already have one and follow the residency program's Twitter and Instagram accounts. Residents were also encouraged to enroll in a private Slack network within the residency program. We surveyed residents in May and June 2020 (pre-intervention) and March 2021 (post-intervention). Analytics from the residency program's social media accounts and Slack were recorded. Data were analyzed using a mixed-methods approach. Results Response rates from residents regarding the impact of social media interventions on education, connectedness, and recruitment were 98% (100/102) pre-intervention and 74.5% (76/102) post-intervention. During the study period from May 2020 to March 2021, chief resident posts on the residency program's Twitter and Instagram accounts garnered 447,467 and 151,341 impressions, respectively. Posts with the highest average impressions were those related to advocacy. After the intervention, residents reported increased connectedness to residents in other classes and increased usage of their personal Twitter and Slack accounts for learning and education. Residents rated the program's Instagram account as a useful recruitment tool. Feasibility of posting was assessed by the number of posts by chief residents during the study period (Twitter n=806, Instagram n=67). There were no costs. Conclusion Our data shows that social media in residency is feasible, cost-effective, and valuable for education, connectedness, and recruitment. We outlined specific ways social media was feasible and useful in these domains.

Feasibility of large-scale eOSCES: the simultaneous evaluation of 500 medical students during a mock examination

The COVID-19 pandemic has led health schools to cancel many on-site training and exams. Teachers were looking for the best option to carry out online OSCEs, and Zoom was the obvious choice since many schools have used it to pursue education purposes.

METHODS

We conducted a feasibility study during the 2020-2021 college year divided into six pilot phases and the large-scale eOSCEs on Zoom on June 30th, 2021. We developed a specific application allowing us to mass create Zoom meetings and built an entire organization, including a technical support system (an SOS room and catching-up rooms) and teachers' training sessions. We assessed satisfaction via an online survey.

RESULTS

On June 30th, 531/794 fifth-year medical students (67%) participated in a large-scale mock exam distributed in 135 Zoom meeting rooms with the mobilization of 298 teachers who either participated in the Zoom meetings as standardized patients (N =135, 45%) or examiners (N =135, 45%) or as supervisors in the catching-up rooms (N =16, 6%) or the SOS room (N =12, 4%). In addition, 32/270 teachers (12%) experienced difficulties connecting to their Zoom meetings and sought the help of an SOS room member. Furthermore, 40/531 students (7%) were either late to their station or had technical difficulties and declared those issues online and were welcomed in one of the catching-up rooms to perform their eOSCE stations. Additionally, 518/531 students (98%) completed the entire circuit of three stations, and 225/531 students (42%) answered the online survey. Among them, 194/225 (86%) found eOSCES helpful for training and expressed their satisfaction with this experience.

CONCLUSION

Organizing large-scale eOSCEs on Zoom is feasible with the appropriate tools. In addition, eOCSEs should be considered complementary to on-site OSCEs and to train medical students in telemedicine.

Virtual Objective Structured Clinical Examination Experiences and Performance in Physical Medicine and Rehabilitation Residency

BACKGROUND

Virtual education has been described before and during the COVID-19 pandemic. Studies evaluating virtual objective structured clinical examinations with postgraduate learners are lacking. This study (1) evaluated the experiences of all participants in a virtual objective structured clinical examination and (2) assessed the validity and reliability of selected virtual objective structured clinical examination stations for skills in physical medicine and rehabilitation.

METHODS

Convergent mixed-methods design was used. Participants included three physical medicine and rehabilitation residency programs holding a joint virtual objective structured clinical examination. Analysis included descriptive statistics and thematic analysis. Performance of virtual to previous in-person objective structured clinical examination was compared using independent t tests.

RESULTS

Survey response rate was 85%. No participants had previous experience with virtual objective structured clinical examination. Participants found the virtual objective structured clinical examination to be acceptable (79.4%), believable (84.4%), and valuable for learning (93.9%). No significant differences between in-person and virtual objective structured clinical examination scores was found for three-fourth stations and improved scores in one fourth. Four themes were identified: (1) virtual objective structured clinical examinations are better for communication stations; (2) significant organization is required to run a virtual objective structured clinical examination; (3) adaptations are required compared with in-person objective structured clinical examinations; and (4) virtual objective structured clinical examinations provide improved accessibility and useful practice for virtual clinical encounters.

CONCLUSIONS

Utility of virtual objective structured clinical examinations as a component of a program of assessment should be carefully considered and may provide valuable learning opportunities going forward.

Experience with an OSCE anamnesis station via Zoom: Feasibility, acceptance and challenges from the perspective of students, simulated patients and examiners during the COVID-19 pandemic

Aim: Assessments of practical clinical competencies pose a challenge during the COVID-19 pandemic. Reports about OSCE stations administered online show that, despite technical feasibility and acceptance, there is a lingering desire for in-person assessments. Barriers and challenges must therefore also be identified in regard to the future integration of digital competencies into the curriculum. Based on a study investigating the feasibility and acceptance of an online OSCE anamnesis station and the descriptions given by students, simulated patients and examiners of the challenges and limitations, we make recommendations for necessary future adaptations to anamnesis training and testing in the context of telemedicine. Method: We surveyed students after completion of an OSCE anamnesis station, adapted to the telemedical setting, that was administered as an alternative assessment to 149 students via Zoom®. Using semi-structured interviews, we analyzed the resulting challenges and limitations as seen by all of the participants. Results: We confirm the existence of good technical and organizational feasibility, positive learning experiences through feedback, the acquisition of clinical competencies, and a high acceptance of this format as an alternative assessment during the pandemic. Using the semi-structured interviews, it was also possible to analyze additional categories that identify necessary adaptations of this type of format. Conclusion: Adaptation of the content-based training for all of the participants and a targeted revision of the checklists, e.g., regarding communication techniques in a telemedicine setting, is required due to the effects of the online format on communication and interactions between students and simulated patients.

Opportunities and Challenges in Medical Education During the COVID-19 Pandemic

The abrupt onset of the pandemic in early 2020 presented a clear challenge to medical and resident education across the nation. Numerous changes were made to allow educational efforts to continue, including the use of virtual formats. The benefits and challenges to virtual learning, as well as the difficulty in transitioning certain skills, such as thorough physical examination and procedural skills, are discussed. Future opportunities exist for hybrid virtual learning and conferences and the development of formal telehealth curricula. The effect of these changes on professional identity formation must be intentionally addressed and role model and mentor relationships fostered both virtually and in person. Given the availability of both vaccines and personal protective equipment, appropriately protected students and trainees should be afforded every opportunity to pursue hands-on medical learning in preparation for their future careers. [Pediatr Ann. 2022;51(8):e319-e323.].

Virtual Skills Training Format for Teaching Intrauterine Contraception Insertion During Coronavirus Disease 2019

The coronavirus disease 2019 pandemic resulted in the curtailment of face-to-face clinical skills training in 2020. To meet Title X workforce needs, the National Clinical Training Center for Family Planning transitioned onsite intrauterine contraception training to a virtual format using online didactic material and livestreamed training and telementoring. Videos demonstrated the placement and removal of intrauterine contraceptives, and all necessary supplies were shipped directly to participants. Attendees reported a high level of skill uptake and impact on their practice. This pilot study suggests that virtual skills training is suitable for providers unable to travel to in-person events.

Virtual clinical assessment in medical education: an investigation of online conference technology

As a result of the Covid-19 pandemic, medical education institutions were suddenly and unexpectedly faced with making significant changes in delivering their clinical assessments to comply with social distancing requirements and limited access to clinical education centres. Seeking a potential solution to these new circumstances, we designed, implemented and evaluated an online virtual OSCE, as a 'proof of concept' intervention study. Our qualitative research involved document analysis of the stages of decision-making and consultation in designing the intervention, and thematic analysis based on the perspectives and experiences of the key stakeholders (final year students, clinical examiners, simulated patients and faculty staff who acted as station assistants), gathered through surveys with Likert-scale questions and free text comments, and online discussion groups which were recorded and transcribed. From our analysis, we identified four themes: optimising assessment design for online delivery, ensuring clinical authenticity, recognising and addressing feelings and apprehensions, and anticipating challenges through incident planning and risk mitigation. Through the data gathered at each stage of the intervention, and the involvement of key stakeholders in the design and evaluation, our study highlights examples of effective practice for future applications of online technologies in assessment, provides guidance for designing and implementing online virtual assessment, and lays a foundation for comparative, longitudinal research on the significant and increasing roles played by technology in healthcare professional education and practice.

Design, Implementation, and Evaluation of a Distance Learning Framework to Adapt to the Changing Landscape of Anatomy Instruction in Medical Education During COVID-19 Pandemic: A Proof-of-Concept Study

This study presents the design of a DL-framework to deliver anatomy teaching that provides a microfiche of the onsite anatomy learning experience during the mandated COVID-19 lockdown. First, using nominal-group technique, we identified the DL learning theories to be employed in blueprinting the DL-framework. Effectiveness of the designed DL-framework in anatomy teaching was demonstrated using the exemplar of the Head and Neck (H&N) course during COVID-19 lockdown, in the pre-clerkship curriculum at our medical school. The dissemination of the DL-framework in the anatomy course was informed by the Analyse, Design, Develop, Implement, and Evaluate (ADDIE) model. The efficiency of the DL-framework was evaluated using the first two levels of Kirkpatrick's model. Versatility of the DL-framework was demonstrated by aligning its precepts with individual domains of key learning outcomes framework. The framework's blueprint was designed amalgamating principles of: Garrison's community inquiry, Siemens' connectivism and Harasim's online-collaborative-learning; and improved using Anderson's DL-model. Following the implementation of the DL-framework in the H&N course informed by ADDIE, the framework's efficiency was evaluated. In total, 70% students responded to the survey assessing perception toward DL (Kirkpatrick's Level: 1). Descriptive analysis of the survey results showed that the DL-framework was positively received by students and attested that students had an enriched learning experience, which promoted collaborative-learning and student-autonomy. For, Kirkpatrick's Level: 2 i.e., cognitive development, we compared the summative assessment performance in the H&N course across three cohort of students. The results show that the scores of the cohort, which experienced the course entirely through DL modality was statistically higher (P < 0.01) than both the other cohorts, indicating that shift to DL did not have an adverse effect on students' learning. Using Bourdieu's Theory of Practice, we showed that the DL-framework is an efficient pedagogical approach, pertinent for medical schools to adopt; and is versatile as it attests to the key domains of students' learning outcomes in the different learning outcomes framework. To our knowledge this is the first-study of its kind where a rationale and theory-guided approach has been availed not only to blueprint a DL framework, but also to implement it in the MBBS curriculum.

Adapting Medical Education Initiatives Through Team-Based e-Learning, Telemedicine Objective Structured Clinical Exams, and Student-Led Community Outreach During the COVID-19 Pandemic

Although the COVID-19 pandemic has quickly prompted medical schools and students around the world to transition from their traditional classrooms to web-based learning, the global crisis has inspired the development of innovative e-learning solutions that use existing technology and other web-based tools to continue nurturing the education of medical students while ensuring the public health and safety of both students and faculty members alike. Through the perspective of medical students, we share how the COVID-19 pandemic has impacted and transformed small team-based learning in medical education; changed objective structured clinical exam evaluations and the practice of clinical skills through telemedicine; and nurtured nationwide, web-based, student-led initiatives for community outreach, telehealth, and medical services.