Using standardized patients to evaluate medical students' evidence-based medicine skills

The Use of Mock Standardized/Simulated Patient Encounters in Facilitating Development of Clinical Competence in Medical Students

Background Standardized patient (SP) encounters are used in medical school to mimic clinical practice by exposing students to possible clinical situations they may encounter in future practice. SP includes trained individuals who portray the roles of patients to help medical students practice recording medical histories, physical exam skills, communication skills, and typing a subjective, objective, assessment and plan (SOAP) note. The goal of SP is to prepare medical students adequately during their didactic years before beginning clerkships. SP encounters have become standard in medical school curriculums, but the option for a mock SP encounter has not.  Methods In this study, a total of 34 participants completed an eight-question survey before a mock SP experience and after their graded SP encounters to assess the students' confidence levels and preparedness. Each question addressed a different aspect of the student's satisfaction and attitude regarding the SP encounters. The answers were measured on a Likert scale from 1 (not prepared) to 5 (very prepared). The central tendency (mean) was analyzed using a T-test with Welch's method. The standard deviation was analyzed using Bonett's test. A Cronbach's alpha was used to show the reliability of the survey used. Results The first four questions addressed the student's satisfaction with their mock SP experience. The mean student satisfaction with being able to ask questions to improve their knowledge and understanding improved, with a p-value of < 0.001. Student satisfaction with having the opportunity to record a patient's history improved, with a p-value of < 0.001. The overall satisfaction with having a chance to practice physical exam skills improved, with a p-value of < 0.001. Mean student satisfaction with practicing treatment and counseling of patients improved, with a p-value of <0.001. The final four questions addressed the students' attitudes regarding their experience. The mean confidence of students improved, with a p-value of <0.001. Students also reported an improved attitude towards the feedback they received, with a p-value of <0.001. The final two questions did not show a statistically significant difference in answers. Students rated the final two questions equally before and after the mock SP experience, with p-values > 0.05. These questions addressed whether mock SP encounters would be beneficial and asked if they wanted additional mock simulation opportunities built into the curriculum. Conclusions The students showed improved confidence, attitudes, and satisfaction surrounding standardized patient encounters. The results supported the hypothesis that there would be a difference between the responses before and after the SP encounter. The questionnaire showed that the students reported subjective changes in their competence following the encounter.

Putting undergraduate medical students in AI-CDSS designers' shoes: An innovative teaching method to develop digital health critical thinking

INTRODUCTION

Digital health programs are urgently needed to accelerate the adoption of Artificial Intelligence and Clinical Decision Support Systems (AI-CDSS) in clinical settings. However, such programs are still lacking for undergraduate medical students, and new approaches are required to prepare them for the arrival of new and unknown technologies. At University Paris Cité medical school, we designed an innovative program to develop the digital health critical thinking of undergraduate medical students that consisted of putting medical students in AI-CDSS designers' shoes.

METHODS

We followed the six steps of Kern's approach for curriculum development: identification of needs, definition of objectives, design of an educational strategy, implementation, development of an assessment and design of program evaluation.

RESULTS

A stand-alone and elective AI-CDSS program was implemented for fourth-year medical students. Each session was designed from an AI-CDSS designer viewpoint, with theoretical and practical teaching and brainstorming time on a project that consisted of designing an AI-CDSS in small groups. From 2021 to 2022, 15 students were enrolled: they rated the program 4.4/5, and 80% recommended it. Seventy-four percent considered that they had acquired new skills useful for clinical practice, and 66% felt more confident with technologies. The AI-CDSS program aroused great enthusiasm and strong engagement of students: 8 designed an AI-CDSS and wrote two scientific 5-page articles presented at the Medical Informatics Europe conference; 4 students were involved in a CDSS research project; 2 students asked for a hospital internship in digital health; and 1 decided to pursue PhD training.

DISCUSSION

Putting students in AI-CDSS designers' shoes seemed to be a fruitful and innovative strategy to develop digital health skills and critical thinking toward AI technologies. We expect that such programs could help future doctors work in rapidly evolving digitalized environments and position themselves as key leaders in digital health.

Development and evaluation of a spiral model of assessing EBM competency using OSCEs in undergraduate medical education

BACKGROUND

Medical students often struggle to understand the relevance of Evidence Based Medicine (EBM) to their clinical practice, yet it is a competence that all students must develop prior to graduation. Objective structured clinical examinations (OSCEs) are a valued assessment tool to assess critical components of EBM competency, particularly different levels of mastery as they progress through the course. This study developed and evaluated EBM based OSCE stations with an aim to establish a spiral approach for EBM OSCE stations for undergraduate medical students.

METHODS

OSCE stations were developed with increasingly complex EBM tasks. OSCE stations were classified according to the classification rubric for EBP assessment tools (CREATE) framework and mapped against the recently published core competencies for evidence-based practice (EBP). Performance data evaluation was undertaken using Classical Test Theory analysing mean scores, pass rates, and station item total correlation (ITC) using SPSS.

RESULTS

Six EBM based OSCE stations assessing various stages of EBM were created for use in high stakes summative OSCEs for different year groups across the undergraduate medical degree. All OSCE stations, except for one, had excellent correlation coefficients and hence a high reliability, ranging from 0.21-0.49. The domain mean score ranged from 13.33 to 16.83 out of 20. High reliability was demonstrated for the each of the summative OSCE circuits (Cronbach's alpha = 0.67-0.85). In the CREATE framework these stations assessed knowledge, skills, and behaviour of medical students in asking, searching, appraising, and integrating evidence in practice. The OSCE stations were useful in assessing six core evidence-based practice competencies, which are meant to be practiced with exercises. A spiral model of OSCEs of increasing complexity was proposed to assess EBM competency as students progressed through the MBChB course.

CONCLUSIONS

The use of the OSCEs is a feasible method of authentically assessing leaner EBM performance and behaviour in a high stakes assessment setting. Use of valid and reliable EBM-based OSCE stations provide evidence for continued development of a hierarchy of assessing scaffolded learning and mastery of EBM competency. Further work is needed to assess their predictive validity.

Medical Student Training in eHealth: Scoping Review

BACKGROUND

eHealth is the use of information and communication technologies to enable and improve health and health care services. It is crucial that medical students receive adequate training in eHealth as they will work in clinical environments that are increasingly being enabled by technology. This trend is especially accelerated by the COVID-19 pandemic as it complicates traditional face-to-face medical consultations and highlights the need for innovative approaches in health care.

OBJECTIVE

This review aims to evaluate the extent and nature of the existing literature on medical student training in eHealth. In detail, it aims to examine what this education consists of, the barriers, enhancing factors, and propositions for improving the medical curriculum. This review focuses primarily on some key technologies such as mobile health (mHealth), the internet of things (IoT), telehealth, and artificial intelligence (AI).

METHODS

Searches were performed on 4 databases, and articles were selected based on the eligibility criteria. Studies had to be related to the training of medical students in eHealth. The eligibility criteria were studies published since 2014, from a peer-reviewed journal, and written in either English or French. A grid was used to extract and chart data.

RESULTS

The search resulted in 25 articles. The most studied aspect was mHealth. eHealth as a broad concept, the IoT, AI, and programming were least covered. A total of 52% (13/25) of all studies contained an intervention, mostly regarding mHealth, electronic health records, web-based medical resources, and programming. The findings included various barriers, enhancing factors, and propositions for improving the medical curriculum.

CONCLUSIONS

Trends have emerged regarding the suboptimal present state of eHealth training and barriers, enhancing factors, and propositions for optimal training. We recommend that additional studies be conducted on the following themes: barriers, enhancing factors, propositions for optimal training, competencies that medical students should acquire, learning outcomes from eHealth training, and patient care outcomes from this training. Additional studies should be conducted on eHealth and each of its aspects, especially on the IoT, AI, programming, and eHealth as a broad concept. Training in eHealth is critical to medical practice in clinical environments that are increasingly being enabled by technology. The need for innovative approaches in health care during the COVID-19 pandemic further highlights the relevance of this training.

Student pharmacists' performance and perceptions on an evidence-based medicine objective structured clinical examination

BACKGROUND AND PURPOSE

Studies have examined evidence-based medicine (EBM) focused objective structured clinical examinations (OSCEs) in medical training, but data are lacking in pharmacy trainees. This study sought to assess student pharmacists' performance on and perceptions of a novel EBM OSCE.

EDUCATIONAL ACTIVITY AND SETTING

This EBM OSCE included answering a drug-information inquiry, researching background questions, calling a simulated provider to acquire specific patient information, developing a foreground clinical question, reviewing pre-appraised trial synopses, and applying evidence to write a recommendation. Pharmacy faculty served as simulated providers and assessed students on knowledge/analytical (AC) and global communication (GC) skills. Students completed a worksheet (WS) that included developing a patient, intervention, comparison, outcome (PICO) statement, trial selection, and clinical recommendation. After OSCE completion, students were surveyed regarding perceptions of their performance and OSCE applicability. Outcomes assessed were performance scores (AC, GC, WS) and student perceptions.

FINDINGS

One-hundred twenty-nine students completed the survey and were included in analysis. AC, WS, and GC performance [median (IQR)] were 75.0 (37.8), 86.4 (36.9), and 88.9 (22.2), respectively, on a 100-point scale. On the WS, 89% of students developed a suitable searchable clinical question and 61% selected the correct trial synopsis to apply to the case. Students felt literature application and WS development were most challenging. A majority of students felt this OSCE increased comfort in engaging with providers (74%) and that these skills correlate with real clinical scenarios (77%).

SUMMARY

OSCEs can be a valuable tool for simulating clinical scenarios and assessing student pharmacists' EBM skills.