1
|
Dong SW, Walker J, Nematollahi S, Nolan NS, Ryder JH. The ID Digital Institute: Building a digital education toolset and community. Transpl Infect Dis 2024; 26:e14297. [PMID: 38884941 DOI: 10.1111/tid.14297] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/30/2024] [Accepted: 05/03/2024] [Indexed: 06/18/2024]
Abstract
Organic digital education (ODE) includes online medical education content that can take various forms, such as blogs, social media, videos, podcasts, or infographics. Multimedia ODE platforms have unique benefits and have quickly become an essential part of medical education. Modern medical educators with competency in digital teaching modalities can leverage these for teaching as well as career development and dissemination of scientific research. The ID Digital Institute is a digital education program with a curriculum designed to equip infectious diseases (ID) professionals with the skills to appraise, create, curate, and integrate ODE into their teaching and career. We share the structure, content, and lessons learned from the ID Digital Institute program. We also illustrate how digital education skills can present unique opportunities to align with current and future transplant and immunocompromised host infectious diseases education efforts.
Collapse
Affiliation(s)
- Sara W Dong
- Division of Infectious Diseases, Departments of Medicine and Pediatrics, Emory University School of Medicine & Children's Healthcare of Atlanta, Atlanta, Georgia, USA
| | - Jeremey Walker
- Division of Infectious Diseases, University of Alabama at Birmingham, Birmingham, Alabama, USA
| | - Saman Nematollahi
- Division of Infectious Diseases, Department of Medicine, University of Arizona College of Medicine, Tucson, Arizona, USA
| | | | - Jonathan H Ryder
- Division of Infectious Diseases, Department of Internal Medicine, University of Nebraska Medical Center, Omaha, Nebraska, USA
| |
Collapse
|
2
|
Grock A, Akuzawa H. With Great Podcasts Comes Great Responsibility. JAMA Netw Open 2024; 7:e2415614. [PMID: 38900430 DOI: 10.1001/jamanetworkopen.2024.15614] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 06/21/2024] Open
Affiliation(s)
- Andrew Grock
- Department of Emergency Medicine, Greater Los Angeles VA Healthcare System, David Geffen School of Medicine at the University of California, Los Angeles
| | - Hiroko Akuzawa
- Department of Emergency Medicine, Greater Los Angeles VA Healthcare System, David Geffen School of Medicine at the University of California, Los Angeles
| |
Collapse
|
3
|
Grock A, Fan T, Berger M, Riddell J. FOAM authorship: Who's teaching our learners? AEM EDUCATION AND TRAINING 2024; 8:e10995. [PMID: 38813090 PMCID: PMC11130452 DOI: 10.1002/aet2.10995] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/05/2024] [Revised: 05/01/2024] [Accepted: 05/02/2024] [Indexed: 05/31/2024]
Abstract
Background Free open-access medical education (FOAM) is extremely popular among learners and educators despite lacking the traditional peer review process. Despite the potential for inaccurate, low-quality, or biased content, little has been published describing FOAM authors. Methods We performed a cross-sectional analysis of 12 months of content from the top 25 blogs in the 2020 Social Media Index from August 2020-2021. We recorded the number of posts per site and descriptive characteristics of authors, including gender affiliation, conflicts of interest (COI) statements, and type of practice (academic, community, or hybrid). Results We identified 2141 posts by 1001 authors. More than half were produced by six websites: EM Docs (266), Life in the Fast Lane (232), EMCrit (188), ALiEM (185), Don't Forget the Bubbles (181), and Rebel EM (174). Most content (1680 posts, 78.5%) lacked a COI statement. Authors were mostly academic (89%), mostly held MD degrees (67.4%), and were mostly men (59.7%). Geographically, most FOAM authors reside in the United States (59.5%), Canada (22.42%), or the United Kingdom (9.4%). Conclusions Of all the posts in the top 25 sites in 2020, more than half came from six sites, and authors were largely North American men in academics with MD degrees. Learners, content creators, and educators should consider the ways in which a more diverse authorship pool might bring value to the FOAM educational experience.
Collapse
Affiliation(s)
- Andrew Grock
- Department of Emergency MedicineDavid Geffen School of Medicine, UCLALos AngelesCaliforniaUSA
- Department of Emergency MedicineGreater Los Angeles VA SystemLos AngelesCaliforniaUSA
| | - Tiffany Fan
- Department of Emergency MedicineHarbor‐UCLA Medical CenterWest CarsonCaliforniaUSA
| | - Max Berger
- Department of Emergency MedicineDavid Geffen School of Medicine, UCLALos AngelesCaliforniaUSA
| | - Jeffrey Riddell
- Clinical Emergency MedicineKeck School of Medicine of the University of Southern CaliforniaCaliforniaLos AngelesUSA
| |
Collapse
|
4
|
Zhao L, Tom S, Patel N, Fermin P, Pedigo R, Bae SW, Jung J, Chan T, Hsiao J. SAEM systematic online academic resource (SOAR) review: Gastrointestinal illnesses. AEM EDUCATION AND TRAINING 2024; 8:e10954. [PMID: 38525362 PMCID: PMC10958398 DOI: 10.1002/aet2.10954] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/30/2023] [Revised: 12/21/2023] [Accepted: 01/04/2024] [Indexed: 03/26/2024]
Abstract
Background and Objectives Free open access medical education (FOAM) has become an essential tool for emergency medicine (EM) education and can be valuable to clinicians as a point-of-care resource. The development of the revised Medical Education Translational Resources Impact and Quality (rMETRIQ) tool provides a standardized means of quality assessment. Previous entries of the Society for Academic Emergency Medicine systematic online academic resource (SOAR) series have focused on renal, endocrine, and sickle cell disorders. In this iteration, we strive to identify, curate, and describe FOAM topics specific to acute gastrointestinal (GI) illnesses. Methods We searched 389 keywords across 11 GI topics that were modified from the 2019 Model of the Clinical Practice of EM (EM Model) using the search engine Google FOAM and within the top 50 websites listed on Academic Life in Emergency Medicine's Social Media Index. The sites underwent preliminary screening to eliminate resources that were not relevant to EM or GI illnesses. Identified resources were evaluated with the rMETRIQ tool by five board-certified EM physicians who received rMETRIQ tool rater training. Results After duplicates of the initial 39,505 resources were eliminated, 8059 remained. Primary screening resulted in a final 1202 resources. The most common categories were large bowel (18%), small bowel (13%), stomach (11%), esophagus (11%), biliary (11%), and liver (10%). Many resources covered multiple topics and subtopics. The final mean intraclass correlation coefficient among the five physicians was 0.95 (95% CI 0.92-0.98) for rMETRIQ scoring. We identified 256 sites considered "high quality" with a rMETRIQ score of 16 or higher as designated in prior reviews. Conclusions This iteration of the SOAR review resulted in the highest number of high-quality resources compared to other SOAR reviews, with 21% of resources thus far scoring ≥ 16. A final list of high-quality resources can guide trainees, educator recommendations, and FOAM authors.
Collapse
Affiliation(s)
- Lisa Zhao
- Department of Emergency Medicine, Greater Los Angeles Veterans Affairs HospitalDavid Geffen School of Medicine, UCLALos AngelesCaliforniaUSA
| | - Sabrina Tom
- Department of Emergency Medicine, Greater Los Angeles Veterans Affairs HospitalDavid Geffen School of Medicine, UCLALos AngelesCaliforniaUSA
| | - Neil Patel
- Department of Emergency Medicine, Greater Los Angeles Veterans Affairs HospitalDavid Geffen School of Medicine, UCLALos AngelesCaliforniaUSA
| | - Patricia Fermin
- Department of Emergency Medicine, Greater Los Angeles Veterans Affairs HospitalDavid Geffen School of Medicine, UCLALos AngelesCaliforniaUSA
| | - Ryan Pedigo
- Department of Emergency Medicine, Harbor–UCLA Medical CenterDavid Geffen School of Medicine, UCLALos AngelesCaliforniaUSA
| | - Shirley Whiinh Bae
- Department of Emergency MedicineLoma Linda University HealthLoma LindaCaliforniaUSA
| | - JooYeon Jung
- Department of OphthalmologyUniversity of CaliforniaDavisCaliforniaUSA
| | - Teresa Chan
- Division of Emergency Medicine, Department of MedicineFaculty of Health Sciences, McMaster UniversityHamiltonOntarioCanada
- McMaster Education, Research, Innovation and Theory ProgramFaculty of Health Sciences, McMaster UniversityHamiltonOntarioCanada
- School of MedicineToronto Metropolitan UniversityTorontoOntarioCanada
| | - Jonie Hsiao
- Department of Emergency Medicine, Greater Los Angeles Veterans Affairs HospitalDavid Geffen School of Medicine, UCLALos AngelesCaliforniaUSA
| |
Collapse
|
5
|
Masters K, Correia R, Nemethy K, Benjamin J, Carver T, MacNeill H. Online learning in health professions education. Part 2: Tools and practical application: AMEE Guide No. 163. MEDICAL TEACHER 2024; 46:18-33. [PMID: 37740948 DOI: 10.1080/0142159x.2023.2259069] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/25/2023]
Abstract
Part 1 of the AMEE Guide Online learning in health professions education focused on foundational concepts such as theory, methods, and instructional design in online learning. Part 2 builds upon Part 1, introducing technology tools and applications of these foundational concepts by exploring the various levels (from beginner to advanced) of utilisation, while describing how their usage can transform Health Professions Education. This Part covers Learning Management Systems, infographics, podcasting, videos, websites, social media, online discussion forums, simulation, virtual patients, extended and virtual reality. Intertwined are other topics, such as online small group teaching, game-based learning, FOAM, online social and collaboration learning, and virtual care teaching. We end by discussing digital scholarship and emerging technologies. Combined with Part 1, the overall aim of Part 2 is to produce a comprehensive overview to help guide effective use online learning in Health Professions Education.
Collapse
Affiliation(s)
- Ken Masters
- Medical Education and Informatics Department, College of Medicine and Health Sciences, Sultan Qaboos University, Sultanate of Oman
| | | | - Kataryna Nemethy
- Baycrest Academy, Dalla Lana School of Public Health, University of Toronto, Toronto, Canada
| | - Jennifer Benjamin
- Department of Education Innovation and Technology, Texas Childrens Hospital (TCH), Texas, USA
| | | | - Heather MacNeill
- Department of Medicine, Continuing Professional Development, University of Toronto, Toronto, Canada
| |
Collapse
|
6
|
Torres-Zegarra BC, Rios-Garcia W, Ñaña-Cordova AM, Arteaga-Cisneros KF, Chalco XCB, Ordoñez MAB, Rios CJG, Godoy CAR, Quezada KLTP, Gutierrez-Arratia JD, Flores-Cohaila JA. Performance of ChatGPT, Bard, Claude, and Bing on the Peruvian National Licensing Medical Examination: a cross-sectional study. JOURNAL OF EDUCATIONAL EVALUATION FOR HEALTH PROFESSIONS 2023; 20:30. [PMID: 37981579 PMCID: PMC11009012 DOI: 10.3352/jeehp.2023.20.30] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/06/2023] [Accepted: 11/07/2023] [Indexed: 11/21/2023]
Abstract
PURPOSE We aimed to describe the performance and evaluate the educational value of justifications provided by artificial intelligence chatbots, including GPT-3.5, GPT-4, Bard, Claude, and Bing, on the Peruvian National Medical Licensing Examination (P-NLME). METHODS This was a cross-sectional analytical study. On July 25, 2023, each multiple-choice question (MCQ) from the P-NLME was entered into each chatbot (GPT-3, GPT-4, Bing, Bard, and Claude) 3 times. Then, 4 medical educators categorized the MCQs in terms of medical area, item type, and whether the MCQ required Peru-specific knowledge. They assessed the educational value of the justifications from the 2 top performers (GPT-4 and Bing). RESULTS GPT-4 scored 86.7% and Bing scored 82.2%, followed by Bard and Claude, and the historical performance of Peruvian examinees was 55%. Among the factors associated with correct answers, only MCQs that required Peru-specific knowledge had lower odds (odds ratio, 0.23; 95% confidence interval, 0.09-0.61), whereas the remaining factors showed no associations. In assessing the educational value of justifications provided by GPT-4 and Bing, neither showed any significant differences in certainty, usefulness, or potential use in the classroom. CONCLUSION Among chatbots, GPT-4 and Bing were the top performers, with Bing performing better at Peru-specific MCQs. Moreover, the educational value of justifications provided by the GPT-4 and Bing could be deemed appropriate. However, it is essential to start addressing the educational value of these chatbots, rather than merely their performance on examinations.
Collapse
Affiliation(s)
| | - Wagner Rios-Garcia
- Sociedad Científica de Estudiantes de Medicina de Ica, Universidad Nacional San Luis Gonzaga, Ica, Peru
| | | | | | | | | | | | | | | | - Jesus Daniel Gutierrez-Arratia
- Academic Department, USAMEDIC, Lima, Peru
- Neurogenetics Research Center, Instituto Nacional de Ciencias Neurologicas, Lima, Peru
| | | |
Collapse
|
7
|
Parga-Belinkie JJ, Cosmini M, Hill D, Berk J, Lockwood K, Patrick M, Novak C, Tarchichi TR. The Role of Medical Education Podcasts in Pediatrics. Pediatrics 2023; 152:e2023062911. [PMID: 37881834 PMCID: PMC10598622 DOI: 10.1542/peds.2023-062911] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 08/08/2023] [Indexed: 10/27/2023] Open
Affiliation(s)
- Joanna J. Parga-Belinkie
- Children’s Hospital of Philadelphia, Philadelphia, Pennsylvania
- University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania
| | | | - David Hill
- University of North Carolina School of Medicine, Chapel Hill, North Carolina
- Campbell School of Osteopathic Medicine, Lillington, North Carolina
| | - Justin Berk
- Brown Med-Peds Program, Providence, Rhode Island
| | - Katie Lockwood
- Children’s Hospital of Philadelphia, Philadelphia, Pennsylvania
- University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania
| | - Michael Patrick
- Nationwide Children’s Hospital, Columbus, Ohio
- The Ohio State University College of Medicine, Columbus, Ohio
| | - Chris Novak
- University of Calgary, Calgary, Alberta, Canada
- Alberta Children’s Hospital, Calgary Alberta, Canada
| | - Tony R. Tarchichi
- University of Pittsburgh School of Medicine, Pittsburgh, Pennsylvania
- UPMC Children’s Hospital of Pittsburgh, Pittsburgh, Pennsylvania
| |
Collapse
|
8
|
Brew-Girard E, Brown R, Salter E, Hattersley C, Hodge O, Leonard X, Macdonald K, Mupanemunda G, Quinn M, Rahman J, Roberts A, Skuse K, Tran M, De Souza S. Hunting for Pearls: A Qualitative Analysis of the Reflections of Students Creating Psychiatric Podcasts. ADVANCES IN MEDICAL EDUCATION AND PRACTICE 2023; 14:1157-1166. [PMID: 37869504 PMCID: PMC10590133 DOI: 10.2147/amep.s411072] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/25/2023] [Accepted: 09/14/2023] [Indexed: 10/24/2023]
Abstract
Purpose This paper aims to explore medical student experiences of creating a peer-to-peer psychiatry educational podcast. Methods During psychiatry placement, ten year-4 University of Bristol medical students created peer-educational multi-episode podcasts on psychiatric topics. Following completion, they submitted reflective essays on their experiences. Qualitative thematic analysis of these essays was completed by two independent authors. Following data familiarisation, authors independently generated codes that were collated into relevant themes. Upon reaching thematic saturation, findings were collated, and member checking was carried out to confirm the validity of findings. Results Themes included effective preparation, choosing content, podcast production, enhancing learning, the weight of responsibility and creating educational support networks. All students found podcast creation to be beneficial for personal learning. Conclusion Exploration of students' experiences creating podcasts can support clearer guidance for medical podcast production, providing opportunities for educators to optimise podcast creation efficiency and educational effectiveness.
Collapse
Affiliation(s)
- Elsa Brew-Girard
- Undergraduate Medical Department, University of Bristol, Bristol, UK
- Adult Psychiatry Department, North Bristol NHS Trust, Bristol, UK
| | - Rachel Brown
- Undergraduate Medical Department, University of Bristol, Bristol, UK
- Psychiatry Postgraduate Medical Education Department, Somerset Partnership NHS Foundation Trust, Taunton, UK
| | - Emma Salter
- Undergraduate Medical Department, University of Bristol, Bristol, UK
- Psychiatry Postgraduate Medical Education Department, Somerset Partnership NHS Foundation Trust, Taunton, UK
| | | | - Oliver Hodge
- Undergraduate Medical Department, University of Bristol, Bristol, UK
| | - Xavier Leonard
- Undergraduate Medical Department, University of Bristol, Bristol, UK
| | - Katie Macdonald
- Undergraduate Medical Department, University of Bristol, Bristol, UK
| | - Grace Mupanemunda
- Undergraduate Medical Department, University of Bristol, Bristol, UK
| | - Marianne Quinn
- Undergraduate Medical Department, University of Bristol, Bristol, UK
| | - Josephine Rahman
- Undergraduate Medical Department, University of Bristol, Bristol, UK
| | - Alice Roberts
- Undergraduate Medical Department, University of Bristol, Bristol, UK
| | - Kate Skuse
- Undergraduate Medical Department, University of Bristol, Bristol, UK
| | - Melanie Tran
- Undergraduate Medical Department, University of Bristol, Bristol, UK
| | - Stephen De Souza
- Undergraduate Medical Department, University of Bristol, Bristol, UK
- Psychiatry Postgraduate Medical Education Department, Somerset Partnership NHS Foundation Trust, Taunton, UK
| |
Collapse
|
9
|
Nagi A, Little A. Needs Assessment of Non-core Content for Podcasts in Emergency Medicine. Cureus 2023; 15:e47613. [PMID: 38021791 PMCID: PMC10667571 DOI: 10.7759/cureus.47613] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 10/24/2023] [Indexed: 12/01/2023] Open
Abstract
Study objective The purpose of this study was to assess the use of podcasts among emergency medicine (EM) students, residents, and attending physicians for non-core content. Methods A survey was administered to medical students interested in emergency medicine, current residents of emergency medicine, and attending physicians to determine which podcasts, if any, they listen to and how often. The purpose of listening to these podcasts was also evaluated and determined it was to learn the non-core content of the specialty. The survey was administered online via multiple platforms including Google Forms (Google, Inc., Mountain View, CA), email, and Twitter (San Francisco, CA). The survey was also distributed via messaging application and word of mouth. Results There were 52 responses that were received on the survey. Of the respondents, 73.1% (N = 38) stated that they do listen to medical podcasts. Of these listeners, 21.2% (N = 11) stated that they listen to them once a year, 15.4% (N = 8) once a month, 11.5% (N = 6) once every two weeks, 19.2% (N = 10) weekly, 11.5% (N = 6) 1-2 days per week, 5.8% (N = 3) 3-4 days per week, 5.8% (N = 3) 5-6 days per week, and 9.6% (N = 5) daily. Based on the 52 responses received, 30.8% (N = 16) stated that they listen to them as a supplement to other sources of foundational content, 25% (N = 13) for general interest, 28.8% (N = 15) for entertainment, 7.7% (N = 4) as a primary resource to learn foundational content in their specialty, and 7.7% (N = 4) other. Based on the data metrics, 51.9% (N = 27) stated that 1-5 of the medical podcasts they listen to discuss the practice of emergency medicine, 44.2% (N = 23) stated that none do, and 3.8% (N = 2) stated that more than 10 do. The list of emergency medicine podcasts being listened to are as follows: EMplify, EM:RAP, EM Over Easy, EM Clerkship, Core EM, EM Basic, EMA, EM Cases, Internet Book of Critical Care, Pharm So Hard, EMCRIT, EM Board Bombs, PEM Rules, and REBEL CAST. When asked how important medical podcasts are for learning within the specialty, 44.2% (N = 23) reported not important, 26.9% (N = 14) reported somewhat important, and 28.8% (N = 15) reported very important. When asked how important medical podcasts are for learning outside the specialty, 40% (N = 20) reported not important, 48% (N = 24) reported somewhat important, and 12% (N = 6) stated very important. When asked if their program includes medical education podcasts as part of their curriculum, 18.4% (N = 9) responded yes, while 81.6% (N = 40) responded no. When asked if they believe medical education podcasts should be included as part of their curriculum, 52% (N = 26) stated yes, while 48% (N = 24) stated no. Conclusion Based on the results of the survey, roughly three-quarters of the surveyees listen to medical podcasts. Of the respondents, 38.5% (N = 20) stated that they listen to these podcasts as some form of supplement to learning content (7.7% (N = 4) primary and 30.8% (N = 16) supplementary), which explains the rising availability of the number of podcasts related to emergency medicine. Of the respondents stating that medical podcasts should be included in their curriculum, 52% (N = 26) signifies the importance of quality learning that these medical podcasts provide to their listeners.
Collapse
Affiliation(s)
- Alvin Nagi
- Emergency Medicine, Kansas Health Science Center - Kansas College of Osteopathic Medicine, Wichita, USA
| | - Andrew Little
- Emergency Medicine, AdventHealth East Orlando, Orlando, USA
| |
Collapse
|
10
|
Khadpe J, Belcher CN, Damore K, Hoglund J, Katirji L, Melton M, Grock A. Emergency Medicine Procedures: A Review of Approved Instructional Resources from the World of Free Open Access Medical Education. Cureus 2023; 15:e45986. [PMID: 37900377 PMCID: PMC10602365 DOI: 10.7759/cureus.45986] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/14/2023] [Accepted: 09/26/2023] [Indexed: 10/31/2023] Open
Abstract
The Academic Life in Emergency Medicine (ALiEM) Approved Instructional Resources (AIR) Series was created in 2014 to address the Free Open Access Medical Education (FOAM) movement's decentralized nature and lack of inherent peer review. The AIR series provides a topic-based, curated list of online educational content vetted by academic emergency medicine (EM) faculty that meets individualized interactive instruction criteria for EM trainees. Relevant FOAM resources were identified from the top 50 FOAM websites using the Social Media Index and then scored by EM faculty using a validated instrument to identify the highest quality posts related to a topic. This article reviews FOAM resources pertaining to EM procedures that were labeled as an "Approved Instructional Resource" or "Honorable Mention" using the AIR series methodology.
Collapse
Affiliation(s)
- Jay Khadpe
- Department of Emergency Medicine, University of Florida College of Medicine - Jacksonville, Jacksonville, USA
| | | | - Katrina Damore
- Department of Emergency Medicine, New York City Health & Hospitals - South Brooklyn Health, Brooklyn, USA
| | - Jessica Hoglund
- Department of Emergency Medicine, Atrium Health, Charlotte, USA
| | - Linda Katirji
- Department of Emergency Medicine, University of Kentucky, Lexington, USA
| | - Matthew Melton
- Department of Emergency Medicine, University of Kentucky, Lexington, USA
| | - Andrew Grock
- Emergency Department, VA Greater Los Angeles Healthcare System, Los Angeles, USA
- Department of Emergency Medicine, University of California, Los Angeles, USA
| |
Collapse
|
11
|
McQuade CN, Simonson MG, Ehrenberger KA, Kohli A. Developing a Web-Based Asynchronous Case Discussion Format on Social Media to Teach Clinical Reasoning: Mixed Methods Study. JMIR MEDICAL EDUCATION 2023; 9:e45277. [PMID: 37556191 PMCID: PMC10448285 DOI: 10.2196/45277] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/22/2022] [Revised: 07/18/2023] [Accepted: 07/18/2023] [Indexed: 08/10/2023]
Abstract
BACKGROUND Case-based learning conferences are valuable to trainees, but growing clinical demands hinder consistent attendance. Social media increasingly acts as a venue for trainees to supplement their education asynchronously. We designed and implemented a web-based asynchronous clinical case discussion series on the Twitter social media platform to fill this educational gap. OBJECTIVE The aim of this mixed methods study is to examine the nature of interactions among web-based case discussion participants and assess local attitudes regarding the educational intervention. METHODS Starting in February 2018, we posted clinical vignettes to a dedicated Twitter account with the prompt "What else do you want to know?" to stimulate discussion. The authors replied in real time when case discussion participants requested additional details. Additional data about the case were posted at regular intervals to the discussion thread to advance the overall case discussion. Participants were asked to explain their reasoning and support their conclusions when appropriate. Web-based engagement was assessed using Twitter Analytics. Participants' posts were qualitatively analyzed for themes, with special attention to examples of using clinical reasoning skills. A codebook of types of participant posts and interactions was refined iteratively. Local engagement and attitudes at our institution were assessed by surveying internal medicine trainees (n=182) and faculty (n=165) after 6 months. RESULTS Over a 6-month period, 11 live case discussions were engaged with by users 1773 times. A total of 86 Twitter profiles spanning 22 US states and 6 countries contributed to discussions among participants and the authors. Participants from all training levels were present, ranging from students to faculty. Interactions among participants and the case moderators were most commonly driven by clinical reasoning, including hypothesis-driven information gathering, discussing the differential diagnosis, and data interpretation or organization. Of 71 respondents to the local survey, 29 (41%) reported having a Twitter account. Of the 29 respondents with Twitter accounts, 17 (59%) reported participating in the case discussions. Respondents agreed that case participation increased both their clinical reasoning skills (15/17, 88%) and clinical knowledge (13/17, 76%). CONCLUSIONS A social media-based serialized case discussion was a feasible asynchronous teaching method for engaging web-based learners of all levels in a clinical reasoning discussion. Further study should examine what factors drive trainee participation in web-based case discussions and under what circumstances asynchronous discussion might be preferred over in-person teaching activities.
Collapse
Affiliation(s)
- Casey N McQuade
- Division of General Internal Medicine, Department of Medicine, University of Pittsburgh School of Medicine, Pittsburgh, PA, United States
| | - Michael G Simonson
- Division of General Internal Medicine, Department of Medicine, University of Pittsburgh School of Medicine, Pittsburgh, PA, United States
| | - Kristen A Ehrenberger
- Division of General Internal Medicine, Department of Medicine, University of Pittsburgh School of Medicine, Pittsburgh, PA, United States
- Department of Pediatrics, University of Pittsburgh School of Medicine, Pittsburgh, PA, United States
| | - Amar Kohli
- Division of General Internal Medicine, Department of Medicine, University of Pittsburgh School of Medicine, Pittsburgh, PA, United States
| |
Collapse
|
12
|
Windram JD, Neal A, McMahon CJ. Evolution in Congenital Cardiology Education: The Rise of Digital-Learning Tools. CJC PEDIATRIC AND CONGENITAL HEART DISEASE 2023; 2:93-102. [PMID: 37970528 PMCID: PMC10642146 DOI: 10.1016/j.cjcpc.2022.12.005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/30/2022] [Accepted: 12/23/2022] [Indexed: 11/17/2023]
Abstract
Technology-enhanced learning is now an established part of medical education due to its ready availability and on-demand nature. This offers new opportunities but also challenges to both learners and teachers. This review outlines the current use of social media tools and online resources in medical education with a particular emphasis on congenital cardiology. It provides strategies to the reader on how to optimize learning in the digital environment and offers guidance on how such tools can be incorporated into routine educational practice. Suggestions for developing and assessing educational material online are discussed. Lastly, the concepts of digital professionalism and digital scholarship are explored to aid medical teachers and educators employ these technologies effectively into their teaching and career development.
Collapse
Affiliation(s)
- Jonathan D. Windram
- Department of Cardiology, Mazankowski Heart Institute, University of Alberta, Edmonton, Alberta, Canada
| | - Ashley Neal
- Department of Pediatric Cardiology, Cincinnati Children’s Hospital, Cincinnati, Ohio, USA
| | - Colin J. McMahon
- Department of Pediatric Cardiology, Children’s Health Ireland, Crumlin, Dublin 12, Ireland
- School of Medicine, University College Dublin, Belfield, Dublin 4, Ireland
- School of Health Professions Education, Maastricht University, Maastricht, the Netherlands
| |
Collapse
|
13
|
Evans FM, Krotinger AA, Lilaonitkul M, Khaled HF, Pereira GA, Staffa SJ, Wolbrink TA. Evaluation of Open Access Websites for Anesthesia Education. Anesth Analg 2022; 135:1233-1244. [PMID: 35983999 DOI: 10.1213/ane.0000000000006183] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/20/2023]
Abstract
BACKGROUND While the prevalence of free, open access medical education resources for health professionals has expanded over the past 10 years, many educational resources for health care professionals are not publicly available or require fees for access. This lack of open access creates global inequities in the availability and sharing of information and may have the most significant impact on health care providers with the greatest need. The extent of open access online educational websites aimed for clinicians and trainees in anesthesiology worldwide is unknown. In this study, we aimed to identify and evaluate the quality of websites designed to provide open access educational resources for anesthesia trainees and clinicians. METHODS A PubMed search of articles published between 2009 and 2020, and a Startpage search engine web search was conducted in May 2021 to identify websites using the following inclusion criteria: (1) contain educational content relevant for anesthesia providers or trainees, (2) offer content free of charge, and (3) are written in the English language. Websites were each scored by 2 independent reviewers using a website quality evaluation tool with previous validity evidence that was modified for anesthesia (the Anesthesia Medical Education Website Quality Evaluation Tool). RESULTS Seventy-five articles and 175 websites were identified; 37 websites met inclusion criteria. The most common types of educational content contained in the websites included videos (66%, 25/37), text-based resources (51%, 19/37), podcasts (35%, 13/37), and interactive learning resources (32%, 12/37). Few websites described an editorial review process (24%, 9/37) or included opportunities for active engagement or interaction by learners (30%,11/37). Scores by tertile differed significantly across multiple domains, including disclosure of author/webmaster/website institution; description of an editorial review process; relevancy to residents, fellows, and faculty; comprehensiveness; accuracy; disclosure of content creation or revision; ease of access to information; interactivity; clear and professional presentation of information; and links to external information. CONCLUSIONS We found 37 open access websites for anesthesia education available on the Internet. Many of these websites may serve as a valuable resource for anesthesia clinicians looking for self-directed learning resources and for educators seeking to curate resources into thoughtfully integrated learning experiences. Ongoing efforts are needed to expand the number and improve the existing open access websites, especially with interactivity, to support the education and training of anesthesia providers in even the most resource-limited areas of the world. Our findings may provide recommendations for those educators and organizations seeking to fill this needed gap to create new high-quality educational websites.
Collapse
Affiliation(s)
- Faye M Evans
- From the Department of Anesthesiology, Critical Care, and Pain Medicine, Boston Children's Hospital, Boston, Massachusetts
- Harvard Medical School, Boston, Massachusetts
| | | | - Maytinee Lilaonitkul
- Department of Anesthesia and Perioperative Care, University of California San Francisco, San Francisco, California
| | - Hiba F Khaled
- Office of Education and Quality Improvement, Harvard Medical School, Boston, Massachusetts
| | - Gretchen A Pereira
- From the Department of Anesthesiology, Critical Care, and Pain Medicine, Boston Children's Hospital, Boston, Massachusetts
- OPENPediatrics, Boston, Massachusetts
| | - Steven J Staffa
- From the Department of Anesthesiology, Critical Care, and Pain Medicine, Boston Children's Hospital, Boston, Massachusetts
| | - Traci A Wolbrink
- From the Department of Anesthesiology, Critical Care, and Pain Medicine, Boston Children's Hospital, Boston, Massachusetts
- Harvard Medical School, Boston, Massachusetts
| |
Collapse
|
14
|
Boreskie PE, Chan TM, Novak C, Johnson A, Wolpaw J, Ong A, Priddis K, Buddhdev P, Adkins J, Silverman JA, Davis T, Siegler JE. Medical Education Blog and Podcast Utilization During the COVID-19 Pandemic. Cureus 2022; 14:e23361. [PMID: 35475051 PMCID: PMC9020462 DOI: 10.7759/cureus.23361] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 03/20/2022] [Indexed: 11/05/2022] Open
Abstract
Introduction The coronavirus disease 2019 (COVID-19) pandemic disrupted traditional in-person learning models. Free Open Access Medical (FOAM) education resources naturally filled this void, so we evaluated how medical blog and podcast utilization changed during the early months of the pandemic. Methods Academic medical podcast and blog producers were surveyed on blog and podcast utilization immediately before (January-March 2020) and after (April-May 2020) the COVID-19 pandemic declaration and subsequent lockdown. Utilization is quantified in terms of blog post pageviews and podcast downloads. Linear regression was used to estimate the effect of publication during the COVID-19 period on 30-day downloads or pageviews. A linear mixed model was developed to confirm this relationship after adjustment for independent predictors of higher 30-day downloads or pageviews, using the podcast or blog as a random intercept. Results Compared to the pre-pandemic period, downloads and pageviews per unique blog and podcast publication significantly increased for blogs (median 30-day pageviews 802 to 1860, p<0.0001) but not for podcasts (median 30-day downloads 2726 to 1781, p=0.27). Publications that contained COVID-19 content were strongly associated with higher monthly utilization (β=7.21, 95% CI 6.29-8.14 p<0.001), and even non-COVID-19 material had higher utilization in the early pandemic (median 30-day downloads/pageviews 868 to 1380, p<0.0001). Discussion The increased blog pageviews during the early months of the COVID-19 pandemic demonstrated the important role of blogs in rapid knowledge translation. Podcasts did not experience a similar increase in utilization.
Collapse
Affiliation(s)
| | - Teresa M Chan
- Emergency Medicine, McMaster University, Hamilton, CAN
| | - Chris Novak
- Pediatrics, University of Calgary, Calgary, CAN
| | - Adam Johnson
- Vascular and Endovascular Surgery, Presbyterian Weill Cornell College of Medicine, New York, USA
| | - Jed Wolpaw
- Anesthesia and Critical Care Medicine, Johns Hopkins University School of Medicine, Baltimore, USA
| | - Andrew Ong
- Gastroenterology and Hepatology, Singapore General Hospital, Singapore, SGP
| | - Katherine Priddis
- Pediatric Emergency Medicine, Watford General Hospital, Watford, GBR
| | - Pranai Buddhdev
- Pediatric Orthopaedics, Broomfield Hospital, Broomfield, GBR
| | - Jessica Adkins
- Emergency Medicine, University of Kentucky, Lexington, USA
| | - Jason A Silverman
- Pediatric Gastroenterology and Nutrition, University of Alberta, Edmonton, CAN
| | - Tessa Davis
- Pediatric Emergency Medicine, Royal London Hospital, London, GBR
| | | |
Collapse
|
15
|
Gray JM, Schnadower D, LaFollette R, Shah AS, Sobolewski B. Going viral: A scoping review of the current state and impact of online research dissemination in emergency medicine. AEM EDUCATION AND TRAINING 2022; 6:e10725. [PMID: 35224409 PMCID: PMC8855629 DOI: 10.1002/aet2.10725] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/15/2021] [Revised: 12/29/2021] [Accepted: 01/04/2022] [Indexed: 05/30/2023]
Abstract
BACKGROUND The use of free open-access medical education (FOAM) and other online knowledge dissemination methods has increased over the past decade. However, the role and impact of these tools in the knowledge translation continuum are poorly understood, potentially limiting the ability of knowledge generators to fully harness and exploit their potential. Here, we aim to comprehensively map and synthesize the literature describing the use of online tools for the dissemination of emergency medicine research. METHODS Using scoping review methodology, we searched the traditional literature via PubMed, CINAHL, EMBASE, ERIC, SCOPUS, and the gray literature for publications exploring online methods to disseminate new research findings. We synthesized the results and constructed a conceptual model of current research dissemination methods. RESULTS We included 79 out of 655 unique abstracts and articles identified in our search, 62 of which were from the traditional literature. We describe six primary domains: integration with traditional literature, measurement of dissemination, online organizations and communities of practice, professional development, quality assurance tools and techniques, and advantages and disadvantages of FOAM. For each domain we present an exemplar article and prevailing gaps in knowledge. Finally, we propose a current conceptual framework for dissemination of new research findings that describes both traditional and novel methods of dissemination. CONCLUSIONS This comprehensive review of the literature and current dissemination framework will empower researchers, research networks, and granting organizations to maximize their use of FOAM and other online methods to disseminate new knowledge as well as provide clinicians a better understanding of the tools and methods by which to access and implement new research findings.
Collapse
Affiliation(s)
- James M. Gray
- Division of Emergency MedicineCincinnati Children’s Hospital Medical CenterCincinnatiOhioUSA
| | - David Schnadower
- Division of Emergency MedicineCincinnati Children’s Hospital Medical CenterCincinnatiOhioUSA
- Department of PediatricsUniversity of Cincinnati College of MedicineCincinnatiOhioUSA
| | - Ryan LaFollette
- Department of Emergency MedicineUniversity of Cincinnati College of MedicineCincinnatiOhioUSA
| | - Ashish S. Shah
- Division of Emergency MedicineRady Children’s HospitalSan DiegoCaliforniaUSA
- Department of PediatricsUniversity of California–San DiegoSan DiegoCaliforniaUSA
| | - Brad Sobolewski
- Division of Emergency MedicineCincinnati Children’s Hospital Medical CenterCincinnatiOhioUSA
- Department of PediatricsUniversity of Cincinnati College of MedicineCincinnatiOhioUSA
| |
Collapse
|
16
|
Maldonado G, Smart J, Wiechmann W, Kaplan SH, Billimek J, Wray A, Toohey S, Boysen-Osborn M. Frequency of Social Media and Digital Scholarship Keywords in U.S. Medical Schools' Promotion and Tenure Guidelines. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2022; 97:105-110. [PMID: 34348378 DOI: 10.1097/acm.0000000000004324] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
PURPOSE Medical schools must have clear policies and procedures for promotion and tenure (P&T) of faculty. Social media and digital scholarship (SMDS) is an emerging form of scholarship capable of reaching audiences quickly, conveniently, and in a wide variety of formats. It is unclear how frequently SMDS is considered during P&T reviews. The authors sought to determine whether current P&T guidelines at medical schools consider SMDS. METHOD The authors acquired P&T guidelines from any U.S. Liaison Committee on Medical Education-accredited medical school (or their governing university) that were available online between October and December 2020. Using an iterative process, they developed a bank of keywords that were specific to SMDS or that could include SMDS between October and December 2020. The authors searched each school's guidelines for each keyword and determined whether the word was being used in relation to crediting faculty for SMDS in the context of P&T procedures. The primary outcome measure was the dichotomous presence or absence of SMDS-specific keywords in each school's P&T guidelines. RESULTS The authors acquired P&T guidelines from 145/154 (94%) medical schools. After removing duplicate documents, the authors considered 139 guidelines. The keyword bank included 59 terms, of which 49 were specific to SMDS and 10 were umbrella terms that could be inclusive of SMDS. Of the 139 guidelines, 121 (87%) contained at least 1 SMDS-specific keyword. Schools had a median of 3 SMDS-specific keywords in their P&T guidelines. CONCLUSIONS As the presence and impact of SMDS increase, schools should provide guidance on its role in the P&T process. Faculty should receive clear guidance on how to document quality SMDS for their promotion file.
Collapse
Affiliation(s)
- Graciela Maldonado
- G. Maldonado is a postgraduate year 2 resident physician, Department of Emergency Medicine, Mayo Clinic, Rochester, Minnesota; ORCID: https://orcid.org/0000-0002-3339-3256
| | - Jonathan Smart
- J. Smart is clinical instructor of emergency medicine and fellow of multimedia design and educational technology, University of California, Irvine School of Medicine, Irvine, California; ORCID: https://orcid.org/0000-0002-9399-8548
| | - Warren Wiechmann
- W. Wiechmann is associate dean for clinical science education and educational technology and associate professor of clinical emergency medicine, University of California, Irvine School of Medicine, Irvine, California; ORCID: https://orcid.org/0000-0002-3701-4356
| | - Sherrie H Kaplan
- S.H. Kaplan is professor of medicine and assistant vice chancellor for healthcare measurement and evaluation, University of California, Irvine School of Medicine, Irvine, California; ORCID: https://orcid.org/0000-0002-8644-5849
| | - John Billimek
- J. Billimek is associate professor of family medicine and vice chair for academic affairs, University of California, Irvine School of Medicine, Irvine, California; ORCID: https://orcid.org/0000-0001-6532-3263
| | - Alisa Wray
- A. Wray is assistant professor of clinical emergency medicine and associate residency program director, University of California, Irvine School of Medicine, Irvine, California; ORCID: https://orcid.org/0000-0002-0059-4895
| | - Shannon Toohey
- S. Toohey is assistant professor of clinical emergency medicine and residency program director, University of California, Irvine School of Medicine, Irvine, California; ORCID: https://orcid.org/0000-0002-1887-633X
| | - Megan Boysen-Osborn
- M. Boysen-Osborn is associate dean for students and associate professor of clinical emergency medicine, University of California, Irvine School of Medicine, Irvine, California, and vice chair for education, Department of Emergency Medicine, University of California, Irvine Medical Center, Orange, California; ORCID: https://orcid.org/0000-0001-6676-6429
| |
Collapse
|
17
|
Morrison Ponce DP, Tomlinson S, Sobolewski B. FOAM Club: A Spin on the Traditional Journal Club Format Focused on Blogs and Podcasts. AEM EDUCATION AND TRAINING 2021; 5:e10516. [PMID: 34027281 PMCID: PMC8122134 DOI: 10.1002/aet2.10516] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/15/2020] [Revised: 07/24/2020] [Accepted: 07/27/2020] [Indexed: 05/21/2023]
Affiliation(s)
- Daphne P. Morrison Ponce
- Department of Emergency MedicineDivision of Pediatric Emergency MedicineUniversity of MichiganAnn ArborMIUSA
- United States Navy Medical CorpsAnn ArborMIUSA
| | - Sarah Tomlinson
- Department of Emergency MedicineDivision of Pediatric Emergency MedicineUniversity of MichiganAnn ArborMIUSA
| | - Brad Sobolewski
- Department of PediatricsUniversity of Cincinnati College of Medicine & Division of Emergency MedicineCincinnati Children’s Hospital Medical CenterCincinnatiOHUSA
| |
Collapse
|
18
|
Grock A, Jordan J, Zaver F, Colmers‐Gray IN, Krishnan K, Chan T, Thoma B. The revised Approved Instructional Resources score: An improved quality evaluation tool for online educational resources. AEM EDUCATION AND TRAINING 2021; 5:e10601. [PMID: 34141997 PMCID: PMC8194147 DOI: 10.1002/aet2.10601] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/28/2021] [Revised: 03/23/2021] [Accepted: 03/26/2021] [Indexed: 05/21/2023]
Abstract
BACKGROUND Free Open-Access Medical education (FOAM) use among residents continues to rise. However, it often lacks quality assurance processes and residents receive little guidance on quality assessment. The Academic Life in Emergency Medicine Approved Instructional Resources tool (AAT) was created for FOAM appraisal by and for expert educators and has demonstrated validity in this context. It has yet to be evaluated in other populations. OBJECTIVES We assessed the AAT's usability in a diverse population of practicing emergency medicine (EM) physicians, residents, and medical students; solicited feedback; and developed a revised tool. METHODS As part of the Medical Education Translational Resources: Impact and Quality (METRIQ) study, we recruited medical students, EM residents, and EM attendings to evaluate five FOAM posts with the AAT and provide quantitative and qualitative feedback via an online survey. Two independent analysts performed a qualitative thematic analysis with discrepancies resolved through discussion and negotiated consensus. This analysis informed development of an initial revised AAT, which was then further refined after pilot testing among the author group. The final tool was reassessed for reliability. RESULTS Of 330 recruited international participants, 309 completed all ratings. The Best Evidence in Emergency Medicine (BEEM) score was the component most frequently reported as difficult to use. Several themes emerged from the qualitative analysis: for ease of use-understandable, logically structured, concise, and aligned with educational value. Limitations include deviation from questionnaire best practices, validity concerns, and challenges assessing evidence-based medicine. Themes supporting its use include evaluative utility and usability. The author group pilot tested the initial revised AAT, revealing a total score average measure intraclass correlation coefficient (ICC) of moderate reliability (ICC = 0.68, 95% confidence interval [CI] = 0 to 0.962). The final AAT's average measure ICC was 0.88 (95% CI = 0.77 to 0.95). CONCLUSIONS We developed the final revised AAT from usability feedback. The new score has significantly increased usability, but will need to be reassessed for reliability in a broad population.
Collapse
Affiliation(s)
- Andrew Grock
- Division of Emergency MedicineGreater Los Angeles VA Healthcare SystemLos AngelesCaliforniaUSA
- David Geffen School of MedicineUCLALos AngelesCaliforniaUSA
| | - Jaime Jordan
- David Geffen School of MedicineUCLALos AngelesCaliforniaUSA
- Ronald Reagan UCLA Medical CenterDavid Geffen School of Medicine at UCLALos AngelesCaliforniaUSA
| | - Fareen Zaver
- Division of Emergency MedicineUniversity of CalgaryCalgaryAlbertaCanada
| | | | - Keeth Krishnan
- Department of Family MedicineMcMaster UniversityHamiltonOntarioCanada
| | - Teresa Chan
- Division of Emergency Medicine and Division of Education & InnovationDepartment of MedicineMcMaster UniversityHamiltonOntarioCanada
- Faculty of Health SciencesMcMaster UniversityHamiltonOntarioCanada
| | - Brent Thoma
- Department of Emergency MedicineUniversity of SaskatchewanSaskatoonSaskatchewanCanada
- Royal College of Physicians and Surgeons of CanadaOttawaOntarioCanada
| |
Collapse
|
19
|
Grock A, Chan W, Aluisio AR, Alsup C, Huang D, Joshi N. Holes in the FOAM: An Analysis of Curricular Comprehensiveness in Online Educational Resources. AEM EDUCATION AND TRAINING 2021; 5:e10556. [PMID: 34124504 PMCID: PMC8171788 DOI: 10.1002/aet2.10556] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/15/2020] [Revised: 11/04/2020] [Accepted: 11/09/2020] [Indexed: 05/05/2023]
Abstract
OBJECTIVES We sought to evaluate Free Open Access Medical Education (FOAM), defined as online educational content available free to anyone, anywhere, at any time, by classifying the most impactful FOAM content per the Social Media Index into the topics and subtopics of the American Board of Emergency Medicine's Model of the Clinical Practice of Emergency Medicine. We then analyzed FOAM's comprehensiveness by describing over- and underrepresentation among these topics and subtopics. METHODS First, we searched for FOAM resources based on the most recent 12 months of relevant content for each organ system from the top 50 Social Media Index sites. Next, we classified all 898 posts into its related topics or subtopics per the American Board of Emergency Medicine's Model of the Clinical Practice of Emergency Medicine. Finally, we analyzed how comprehensively FOAM covered each organ system and the frequency of posts that covered each organ system subtopic as well as identified the subtopics with the most frequent coverage. RESULTS The search yielded 898 FOAM posts, of which cardiology and neurology were significantly overrepresented and psychobehavioral; obstetrics and gynecology; and head, ears, eyes, nose, and throat were significantly underrepresented. Among subtopics, acute coronary syndrome had the highest subtopic coverage consisting of 55.5% of all cardiology content. Other highly represented subtopics include renal colic; diabetic ketoacidosis; sepsis; and stroke with 39, 40, 40, and 71% of each of their topic's content, respectively. CONCLUSIONS Although residents and programs are frequently incorporating FOAM into the educational curriculum, these materials seem to lack comprehensiveness. Educators and learners must be aware of these deficits in creating comprehensive emergency medicine curricula.
Collapse
Affiliation(s)
- Andrew Grock
- From theDepartment of Emergency MedicineUniversity of California Los AngelesLos AngelesCAUSA
- and theVA Greater Los Angeles Healthcare System Wadsworth Anaerobe LaboratoryLos AngelesCAUSA
| | - Wendy Chan
- theEmergency DepartmentMaimonides Medical CenterBrooklynNYUSA
| | - Adam R. Aluisio
- theDepartment of Emergency MedicineBrown University Warren Alpert Medical SchoolProvidenceNYUSA
| | - Carl Alsup
- Sierra Nevada Memorial HospitalGrass ValleyCAUSA
| | - Delphine Huang
- UCSF/ZSFGH Department of Emergency Medicine and Kaiser PermanenteSan FranciscoCAUSA
| | - Nikita Joshi
- andAlameda HospitalAlameda Health SystemOaklandCAUSA
| |
Collapse
|
20
|
Ting DK, Boreskie P, Luckett-Gatopoulos S, Gysel L, Lanktree MB, Chan TM. Quality Appraisal and Assurance Techniques for Free Open Access Medical Education (FOAM) Resources: A Rapid Review. Semin Nephrol 2021; 40:309-319. [PMID: 32560781 DOI: 10.1016/j.semnephrol.2020.04.011] [Citation(s) in RCA: 17] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/22/2022]
Abstract
Free open access medical education (FOAM) has disrupted traditional modes of knowledge translation and dissemination. These are popular resources with a wide educational reach. Nephrology has been a leader in FOAM, but many skeptics still question the accuracy and reliability of this content. Recently, quality-assurance techniques have been developed to address these concerns. These techniques may be helpful for readers to appraise the online literature and for institutions to reward the production of high-quality open educational resources. We performed a rapid review of the literature. A medical librarian conducted a systematic search of the Medline and Cumulative Index of Nursing and Allied Health Literature databases. Two independent assessors screened and selected articles, performed a hand-search of reference lists, and scored articles on their quality using the Medical Education Research Study Quality Instrument. Thirteen reports were included for the final descriptive analysis. We identified 10 quality-assessment techniques, and 4 of them having been validated. The quality of the reports was fairly high, with an average Medical Education Research Study Quality Instrument score of 11.5 of 18 (SD, 2.3; range, 7.25-14.25). The calculated Cronbach α was 0.85. There is burgeoning literature on the topic of critical appraisal of open educational resources, and, more specifically, FOAM resources. Many of the techniques used are of varying quality and developed with different intended uses and audiences. By continuing to refine these tools, we can continue not only to support and legitimize the FOAM movement, but also foster individual critical appraisal skills that increasingly are necessary in this age of information.
Collapse
Affiliation(s)
- Daniel K Ting
- Department of Emergency Medicine, University of British Columbia, Vancouver, BC, Canada
| | - Patrick Boreskie
- Department of Emergency Medicine, Max Rady College of Medicine, University of Manitoba, Winnipeg, MB, Canada
| | - S Luckett-Gatopoulos
- Division of Emergency Medicine, Department of Medicine, Western University, London, ON, Canada; Division of Paediatric Emergency Medicine, Department of Paediatrics, McMaster University, Hamilton, ON, Canada
| | - Lisa Gysel
- Interior Health, Royal Inland Hospital Library, Kamloops, BC, Canada
| | - Matthew B Lanktree
- Division of Nephrology, Department of Medicine, McMaster University, Hamilton, ON, Canada
| | - Teresa M Chan
- Division of Emergency Medicine, Department of Medicine, McMaster University, Hamilton, ON, Canada; Department of Emergency Medicine, Hamilton General Hospital, Hamilton Health Sciences Centre, Hamilton, ON, Canada; Program for Faculty Development, Faculty of Health Sciences, McMaster University, Hamilton, ON, Canada; McMaster Education Research, Innovation, and Theory Program (MERIT), McMaster University, Hamilton, ON, Canada.
| |
Collapse
|
21
|
Prager R, Pratte MT, Unni RR, Bala S, Ng Fat Hing N, Wu K, McGrath TA, Thomas A, Thoma B, Kyeremanteng K. Content Analysis and Characterization of Medical Tweets During the Early Covid-19 Pandemic. Cureus 2021; 13:e13594. [PMID: 33815994 PMCID: PMC8007019 DOI: 10.7759/cureus.13594] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/08/2023] Open
Abstract
Objective The novel coronavirus disease 2019 (Covid-19) has infected millions worldwide and impacted the lives of many folds more. Many clinicians share new Covid-19-related resources, research, and ideas within the online Free Open Access to Medical Education (FOAM) community of practice. This study provides a detailed content and contributor analysis of Covid-19-related tweets among the FOAM community during the first months of the pandemic. Design, Setting, and Participants In this social media content analysis study, Twitter was searched from November 1, 2019, to March 21, 2020, for English tweets discussing Covid-19 in the FOAM community. Tweets were classified into one of 13 pre-specified content categories: original research, editorials, FOAM resource, public health, podcast or video, learned experience, refuting false information, policy discussion, emotional impact, blatantly false information, other Covid-19, and non-Covid-19. Further analysis of linked original research and FOAM resources was performed. One-thousand (1000) randomly selected contributor profiles and those deemed to have contributed false information were analyzed. Results The search yielded 8541 original tweets from 4104 contributors. The number of tweets in each content category were: 1557 other Covid-19 (18.2%), 1190 emotional impact (13.9%), 1122 FOAM resources (13.1%), 1111 policy discussion (13.0%), 928 advice (10.9%), 873 learned experience (10.2%), 424 non-Covid-19 (5.0%), 410 podcast or video (4.8%), 304 editorials (3.6%), 275 original research (3.2%), 245 public health (2.9%), 83 refuting false information (1.0%), and 19 blatantly false (0.2%). Conclusions Early in the Covid-19 pandemic, the FOAM community used Twitter to share Covid-19 learned experiences, online resources, crowd-sourced advice, and research and to discuss the emotional impact of Covid-19. Twitter also provided a forum for post-publication peer review of new research. Sharing blatantly false information within this community was infrequent. This study highlights several potential benefits from engaging with the FOAM community on Twitter.
Collapse
Affiliation(s)
- Ross Prager
- Department of Medicine, University of Ottawa, Ottawa, CAN
| | | | - Rudy R Unni
- Department of Medicine, University of Ottawa, Ottawa, CAN
| | - Sudarshan Bala
- Department of Medicine, McMaster University, Hamilton, CAN
| | | | - Kay Wu
- Department of Medicine, McMaster University, Hamilton, CAN
| | | | - Adam Thomas
- Division of Critical Care, University of British Columbia, Vancouver, CAN
| | - Brent Thoma
- Emergency Medicine, University of Saskatchewan, Saskatoon, CAN
| | | |
Collapse
|
22
|
Woods JM, Chan TM, Roland D, Riddell J, Tagg A, Thoma B. Evaluating the reliability of gestalt quality ratings of medical education podcasts: A METRIQ study. PERSPECTIVES ON MEDICAL EDUCATION 2020; 9:302-306. [PMID: 32495235 PMCID: PMC7550476 DOI: 10.1007/s40037-020-00589-x] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
INTRODUCTION Podcasts are increasingly being used for medical education. Studies have found that the assessment of the quality of online resources can be challenging. We sought to determine the reliability of gestalt quality assessment of education podcasts in emergency medicine. METHODS An international, interprofessional sample of raters was recruited through social media, direct contact, and the extended personal network of the study team. Each participant listened to eight podcasts (selected to include a variety of accents, number of speakers, and topics) and rated the quality of that podcast on a seven-point Likert scale. Phi coefficients were calculated within each group and overall. Decision studies were conducted using a phi of 0.8. RESULTS A total of 240 collaborators completed all eight surveys and were included in the analysis. Attendings, medical students, and physician assistants had the lowest individual-level variance and thus the lowest number of required raters to reliably evaluate quality (phi >0.80). Overall, 20 raters were required to reliably evaluate the quality of emergency medicine podcasts. DISCUSSION Gestalt ratings of quality from approximately 20 health professionals are required to reliably assess the quality of a podcast. This finding should inform future work focused on developing and validating tools to support the evaluation of quality in these resources.
Collapse
Affiliation(s)
- Jason M Woods
- Section of Emergency Medicine, Children's Hospital Colorado, University of Colorado School of Medicine, Aurora, Colorado, USA
| | - Teresa M Chan
- Division of Emergency Medicine, Department of Medicine, McMaster University, Hamilton, ON, Canada
| | - Damian Roland
- Paediatric Emergency Medicine, Leicester Royal Infirmary, University of Leicester, Leicester, UK
| | - Jeff Riddell
- Department of Clinical Emergency Medicine, Keck School of Medicine, University of Southern California, Los Angeles, USA
| | - Andrew Tagg
- University of Melbourne, Melbourne, Australia
- Footscray Hospital, Melbourne, Australia
| | - Brent Thoma
- Department of Emergency Medicine, University of Saskatchewan, Saskatoon, Saskatchewan, Canada.
| |
Collapse
|
23
|
Effect of Interpolated Questions on Podcast Knowledge Acquisition and Retention: A Double-Blind, Multicenter, Randomized Controlled Trial. Ann Emerg Med 2020; 76:353-361. [DOI: 10.1016/j.annemergmed.2020.01.021] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/19/2019] [Revised: 11/18/2019] [Accepted: 01/16/2020] [Indexed: 11/22/2022]
|
24
|
Husain A, Repanshek Z, Singh M, Ankel F, Beck-Esmay J, Cabrera D, Chan TM, Cooney R, Gisondi M, Gottlieb M, Khadpe J, Repanshek J, Mason J, Papanagnou D, Riddell J, Trueger NS, Zaver F, Brumfield E. Consensus Guidelines for Digital Scholarship in Academic Promotion. West J Emerg Med 2020; 21:883-891. [PMID: 32726260 PMCID: PMC7390542 DOI: 10.5811/westjem.2020.4.46441] [Citation(s) in RCA: 41] [Impact Index Per Article: 10.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/06/2020] [Accepted: 04/09/2020] [Indexed: 01/10/2023] Open
Abstract
Introduction As scholarship moves into the digital sphere, applicant and promotion and tenure (P&T) committee members lack formal guidance on evaluating the impact of digital scholarly work. The P&T process requires the appraisal of individual scholarly impact in comparison to scholars across institutions and disciplines. As dissemination methods evolve in the digital era, we must adapt traditional P&T processes to include emerging forms of digital scholarship. Methods We conducted a blended, expert consensus procedure using a nominal group process to create a consensus document at the Council of Emergency Medicine Residency Directors Academic Assembly on April 1, 2019. Results We discussed consensus guidelines for evaluation and promotion of digital scholarship with the intent to develop specific, evidence-supported recommendations to P&T committees and applicants. These recommendations included the following: demonstrate scholarship criteria; provide external evidence of impact; and include digital peer-review roles. As traditional scholarship continues to evolve within the digital realm, academic medicine should adapt how that scholarship is evaluated. P&T committees in academic medicine are at the epicenter for supporting this changing paradigm in scholarship. Conclusion P&T committees can critically appraise the quality and impact of digital scholarship using specific, validated tools. Applicants for appointment and promotion should highlight and prepare their digital scholarship to specifically address quality, impact, breadth, and relevance. It is our goal to provide specific, timely guidance for both stakeholders to recognize the value of digital scholarship in advancing our field.
Collapse
Affiliation(s)
- Abbas Husain
- Staten Island University Hospital - Northwell Health, Department of Emergency Medicine, Staten Island, New York
| | - Zachary Repanshek
- Lewis Katz School of Medicine at Temple University, Department of Emergency Medicine, Philadelphia, Pennsylvania
| | - Manpreet Singh
- University of California, Los Angeles Medical School, Department of Emergency Medicine, Los Angeles, California
| | - Felix Ankel
- University of Minnesota Medical School, Department of Emergency Medicine, Minneapolis, Minnesota
| | - Jennifer Beck-Esmay
- Mount Sinai St. Luke's-West, Department of Emergency Medicine, New York, New York
| | - Daniel Cabrera
- Mayo Clinic College of Medicine, Department of Emergency Medicine, Rochester, Minnesota
| | - Teresa M Chan
- McMaster University, Department of Medicine, Hamilton, Ontario
| | - Robert Cooney
- Geisinger Medical Center, Department of Emergency Medicine, Danville, Pennsylvania
| | - Michael Gisondi
- Stanford University School of Medicine, Department of Emergency Medicine, Stanford, California
| | - Michael Gottlieb
- Rush University Medical Center, Department of Emergency Medicine, Chicago, Illinois
| | - Jay Khadpe
- University of Florida College of Medicine, Department of Emergency Medicine, Jacksonville, Florida
| | - Jennifer Repanshek
- Lewis Katz School of Medicine at Temple University, Department of Emergency Medicine, Philadelphia, Pennsylvania
| | - Jessica Mason
- University of San Francisco-Fresno, Department of Emergency Medicine, Fresno, California
| | - Dimitrios Papanagnou
- Sidney Kimmel Medical College at Thomas Jefferson University, Department of Emergency Medicine, Philadelphia, Pennsylvania
| | - Jeff Riddell
- University of Southern California Keck School of Medicine, Department of Emergency Medicine, Los Angeles, California
| | - N Seth Trueger
- Lewis Katz School of Medicine at Temple University, Department of Emergency Medicine, Philadelphia, Pennsylvania.,Stanford University School of Medicine, Department of Emergency Medicine, Stanford, California
| | - Fareen Zaver
- Rush University Medical Center, Department of Emergency Medicine, Chicago, Illinois.,University of California, Los Angeles Medical School, Department of Emergency Medicine, Los Angeles, California
| | - Emily Brumfield
- Oschner Clinic Foundation, Department of Emergency Medicine, New Orleans, Louisiana
| |
Collapse
|
25
|
Greenberger SM, Finnell JT, Chang BP, Garg N, Quinn SM, Bird S, Diercks DB, Doty CI, Gallahue FE, Moreira ME, Ranney ML, Rives L, Kessler CS, Lo B, Schmitz G. Changes to the ACGME Common Program Requirements and Their Potential Impact on Emergency Medicine Core Faculty Protected Time. AEM EDUCATION AND TRAINING 2020; 4:244-253. [PMID: 32704594 PMCID: PMC7369497 DOI: 10.1002/aet2.10421] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/10/2019] [Revised: 11/18/2019] [Accepted: 11/20/2019] [Indexed: 06/11/2023]
Abstract
The Accreditation Council for Graduate Medical Education (ACGME), which regulates residency and fellowship training in the United States, recently revised the minimum standards for all training programs. These standards are codified and published as the Common Program Requirements. Recent specific revisions, particularly removing the requirement ensuring protected time for core faculty, are poised to have a substantial impact on emergency medicine training programs. A group of representatives and relevant stakeholders from national emergency medicine (EM) organizations was convened to assess the potential effects of these changes on core faculty and the training of emergency physicians. We reviewed the literature and results of surveys conducted by EM organizations to examine the role of core faculty protected time. Faculty nonclinical activities contribute greatly to the academic missions of EM training programs. Protected time and reduced clinical hours allow core faculty to engage in education and research, which are two of the three core pillars of academic EM. Loss of core faculty protected time is expected to have detrimental impacts on training programs and on EM generally. We provide consensus recommendations regarding EM core faculty clinical work hour limitations to maintain protected time for educational activities and scholarship and preserve the quality of academic EM.
Collapse
Affiliation(s)
- Sarah M. Greenberger
- Department of Emergency MedicineUniversity of Arkansas for Medical SciencesLittle RockAR
| | - John T. Finnell
- The Regenstrief InstituteIndiana University School of MedicineIndianapolisIN
| | - Bernard P. Chang
- Department of Emergency MedicineColumbia University Medical CenterNew YorkNY
| | - Nidhi Garg
- Department of Emergency MedicineSouthside HospitalNew Hyde ParkNY
| | - Shawn M. Quinn
- Department of Emergency MedicineLehigh Valley Health NetworkAllentownPA
| | - Steven Bird
- Department of Emergency MedicineUniversity of MassachusettsWorcesterMA
| | - Deborah B. Diercks
- Department of Emergency MedicineUniversity of Texas Southwestern Medical CenterDallasTX
| | | | - Fiona E. Gallahue
- Harborview Medical CenterDepartment of Emergency MedicineThe University of WashingtonSeattleWA
| | - Maria E. Moreira
- Department of Emergency MedicineDenver Health and Hospital AuthorityDenverCO
| | | | - Loren Rives
- American College of Emergency PhysiciansIrvingTX
| | | | - Bruce Lo
- Eastern Virginia Medical SchoolNorfolkVA
| | - Gillian Schmitz
- San Antonio Military Medical CenterUniformed Services University of the Health SciencesSan AntonioTX
| |
Collapse
|
26
|
Abstract
Free Open Access Medical education (FOAM) has taken the emergency medicine and critical care worlds by storm in the past decade. This article represents one perspective on the stages of transition for FOAM from its humble beginnings as a grassroots movement to the more recent multiauthor blogs that are described in the peer-reviewed literature. In this article, the authors describe the following four distinct waves of people within the movement, with each wave creating a new stage in the evolution of the FOAM community: Creation by the Founders, Adoption by the Enthusiasts, Structure and Formalization by the Structuralists, and Engagement and Activity by the End Users. The authors contextualize some of the phenomena that have been observed within this field and highlight challenges for the field moving forward.
Collapse
|
27
|
Chan TM, Sebok-Syer SS, Sampson C, Monteiro S. The Quality of Assessment of Learning (Qual) Score: Validity Evidence for a Scoring System Aimed at Rating Short, Workplace-Based Comments on Trainee Performance. TEACHING AND LEARNING IN MEDICINE 2020; 32:319-329. [PMID: 32013584 DOI: 10.1080/10401334.2019.1708365] [Citation(s) in RCA: 20] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
Construct: This study seeks to determine validity evidence for the Quality of Assessment for Learning score (QuAL score), which was created to evaluate short qualitative comments that are related to specific scores entered into a workplace-based assessment, common within the competency-based medical education (CBME) context. Background: In the age of CBME, qualitative comments play an important role in clarifying the quantitative scores rendered by observers at the bedside. Currently there are few practical tools that evaluate mixed data (e.g. associated score-and-comment data), other than the comprehensive Completed Clinical Evaluation Report Rating tool (CCERR) that was originally derived to rate end-of-rotation reports. Approach: A multi-center, randomized cohort-based rating exercise was conducted to evaluate the rating properties of the QuAL score as compared to the CCERR. One group rated comments using the QuAL score, and the other group rated comments using the CCERR. A generalizability study (G-Study) and a decision study (D-study) were conducted to determine the number of meta-raters for a reliable rating (phi-coefficient target of >0.80). Both scores were correlated against rater's gestalt perceptions of utility for both faculty and residents reading the scores. Results: Twenty-five meta-raters from 20 sites participated in this rating exercise. The G-study revealed that the CCERR group (n = 13) rated the comments with a very high reliability (Phi = 0.97). Meanwhile, the QuAL group (n = 12) rated the comments with a similarly high reliability (Phi = 0.97). The QuAL score required only two raters to reach an acceptable target reliability of >0.80, while the CCERR required three. The QuAL score correlated with perceptions of utility (Meta-rater usefulness, Pearson's r = 0.69, p < 0.001; Perceived usefulness for trainee, r = 0.74, p < 0.001). The CCERR performed similarly, correlating with perceived faculty (r = 0.67, <0.001) and resident utility (0.79, <0.001). Conclusions: The QuAL score is reliable rating score that correlates well with perceptions of utility. The QuAL score may be useful for rating shorter comments generated by workplace-based assessments.
Collapse
Affiliation(s)
- Teresa M Chan
- Division of Emergency Medicine, McMaster University, Hamilton, Ontario, Canada
| | | | - Christopher Sampson
- Department of Emergency Medicine, University of Missouri, Columbia, Missouri, USA
| | - Sandra Monteiro
- Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Ontario, Canada
| |
Collapse
|
28
|
Ediger D, Sumpter R, Bridwell RE, Belcher CN. Academic Life in Emergency Medicine (ALiEM) Blog and Podcast Watch: Infectious Diseases. Cureus 2020; 12:e7674. [PMID: 32426186 PMCID: PMC7228787 DOI: 10.7759/cureus.7674] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2022] Open
Abstract
The Academic Life in Emergency Medicine (ALiEM) Approved Instructional Resources (AIR) Series and Approved Instructional Resources - Professional (AIR-Pro) Series were created in 2014 and 2015, respectively, in response to the growing need to curate online educational content as well as create a nationally available curriculum that meets individualized interactive instruction criteria for emergency medicine (EM) trainees. These two online series identify high-quality educational blog and podcast content using an expert-based approach. The AIR series is a continuously building curriculum originally based on the Council of Emergency Medicine Directors (CORD) testing schedule. In September 2019, 61 blog posts and podcasts published within the previous 12 months and relevant to infectious diseases were evaluated by eight attending physicians using the ALiEM AIR scoring instrument. In this review, we summarize the accredited posts on infectious diseases meeting our a priori quality criteria per evaluation by the reviewers.
Collapse
Affiliation(s)
- David Ediger
- Department of Military and Emergency Medicine, Uniformed Services University of the Health Sciences, F. Edward Hébert School of Medicine, Bethesda, USA
| | - Ryan Sumpter
- Department of Military and Emergency Medicine, Uniformed Services University of the Health Sciences, F. Edward Hébert School of Medicine, Bethesda, USA
| | - Rachel E Bridwell
- Department of Emergency Medicine, Brooke Army Medical Center, Fort Sam Houston, USA
| | | |
Collapse
|
29
|
Cardinal P, Barton G, DesRosier K, Yamashita S, Landriault A, Sarti A, Sutherland S, Brien S, McCarragher K, Witter T. Mapping the Expert Mind: Integration Method for Revising the ACES Medical Simulation Curriculum. JOURNAL OF MEDICAL EDUCATION AND CURRICULAR DEVELOPMENT 2020; 7:2382120520913270. [PMID: 32699819 PMCID: PMC7357020 DOI: 10.1177/2382120520913270] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
PURPOSE This article shares our experience developing an integrated curriculum for the ACES (Acute Critical Event Simulation) program. The purpose of the ACES program is to ensure that health care providers develop proficiency in the early management of critically ill patients. The program includes multiple different types of educational interventions (mostly simulation-based) and targets both specialty and family physicians practicing in tertiary and community hospitals. METHODS To facilitate integration between different educational interventions, we developed a knowledge repository consisting of cognitive sequence maps that make explicit the flow of cognitive activities carried out by experts facing different situations - the sequence maps then serving as the foundation upon which multimodal simulation scenarios would be built. To encourage participation of experts, we produced this repository as a peer-reviewed ebook. Five national organizations collaborated with the Royal College of Physicians and Surgeons of Canada to identify and recruit expert authors and reviewers. Foundational chapters, centered on goals/interventions, were first developed to comprehensively address most tasks conducted in the early management of a critically ill patient. Tasks from the foundational chapters were then used to complete the curriculum with situations. The curriculum development consisted of two-phases each followed by a peer-review process. In the first phase, focus groups using web-conferencing were conducted to map clinical practice approaches and in the second, authors completed the body of the chapter (e.g., introduction, definition, concepts, etc.) then provided a more detailed description of each task linked to supporting evidence. RESULTS Sixty-seven authors and thirty-five peer reviewers from various backgrounds (physicians, pharmacists, nurses, respiratory therapists) were recruited. On average, there were 32 tasks and 15 situations per chapter. The average number of focus group meetings needed to develop a map (one map per chapter) was 6.7 (SD ± 3.6). We found that the method greatly facilitated integration between different chapters especially for situations which are not limited to a single goal or intervention. For example, almost half of the tasks of the Hypercapnic Ventilatory Failure chapter map were borrowed from other maps with some modifications, which significantly reduced the authors' workload and enhanced content integration. This chapter was also linked to 6 other chapters. CONCLUSIONS To facilitate curriculum integration, we have developed a knowledge repository consisting of cognitive maps which organize time-sensitive tasks in the proper sequence; the repository serving as the foundation upon which other educational interventions are then built. While this methodology is demanding, authors welcomed the challenge given the scholarly value of their work, thus creating an interprofessional network of educators across Canada.
Collapse
Affiliation(s)
- Pierre Cardinal
- Department of Critical Care Medicine, The Ottawa Hospital, Ottawa, ON, Canada
- Practice, Performance and Innovation, Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
| | - Glenn Barton
- Practice, Performance and Innovation, Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
| | - Kirk DesRosier
- Practice, Performance and Innovation, Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
| | - Sharon Yamashita
- Department of Pharmacy, Sunnybrook Health Sciences Centre, Toronto, ON, Canada
| | - Angèle Landriault
- Practice, Performance and Innovation, Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
| | - Aimee Sarti
- Department of Critical Care Medicine, The Ottawa Hospital, Ottawa, ON, Canada
| | | | - Susan Brien
- Practice, Performance and Innovation, Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
| | - Kevin McCarragher
- Practice, Performance and Innovation, Royal College of Physicians and Surgeons of Canada, Ottawa, ON, Canada
| | - Tobias Witter
- Department of Critical Care and Department of Anesthesiology, Perioperative and Pain Medicine, Dalhousie University, Halifax, NS, Canada
| |
Collapse
|
30
|
Chan TM, Bhalerao A, Thoma B, Trueger NS, Grock A. Thinking Critically About Appraising FOAM. AEM EDUCATION AND TRAINING 2019; 3:398-402. [PMID: 31637359 PMCID: PMC6795350 DOI: 10.1002/aet2.10352] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/01/2019] [Accepted: 04/16/2019] [Indexed: 05/30/2023]
Affiliation(s)
- Teresa M. Chan
- Department of MedicineDivision of Emergency MedicineMcMaster University, and the McMaster Education Research, Innovation, and Theory ProgramHamiltonOntarioCanada
| | - Anuja Bhalerao
- School of MedicineUniversity of TorontoTorontoOntarioCanada
| | - Brent Thoma
- Department of Emergency MedicineUniversity of SaskatchewanSaskatoonSaskatchewanCanada
| | - N. Seth Trueger
- Department of Emergency MedicineNorthwestern UniversityChicagoIL
| | - Andrew Grock
- Division of Emergency MedicineGreater Los Angeles VA Healthcare System, and the Department of Emergency MedicineDavid Geffen School of MedicineUCLALos AngelesCA
| |
Collapse
|
31
|
Grock A, Bhalerao A, Chan TM, Thoma B, Wescott AB, Trueger NS. Systematic Online Academic Resource (SOAR) Review: Renal and Genitourinary. AEM EDUCATION AND TRAINING 2019; 3:375-386. [PMID: 31637355 PMCID: PMC6795358 DOI: 10.1002/aet2.10351] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/03/2019] [Revised: 03/01/2019] [Accepted: 03/06/2019] [Indexed: 05/15/2023]
Abstract
BACKGROUND Online resources for emergency medicine (EM) trainees and physicians have variable quality and inconsistent coverage of core topics. In this first entry of the Society for Academic Emergency Medicine Systematic Online Academic Resource (SOAR) series, we describe the application of a systematic methodology to comprehensively identify, collate, and curate online content for topic-specific modules. METHODS A list of module topics and related terms was generated from the American Board of Emergency Medicine's Model of the Clinical Practice of Emergency Medicine. The authors selected "renal and genitourinary" for the first module, which contained 35 terms; all MeSH headers and colloquial synonyms related to the topic and related terms were searched both within the 100 most impactful online educational websites per the Social Media Index and the FOAMsearch.net search engine. Duplicate entries, journal articles, images, and archives were excluded. The quality of each article was rated using the revised METRIQ (rMETRIQ) score. RESULTS The search yielded 13,058 online resources. After 12,717 items were excluded, 341 underwent quality assessment. All renal/genitourinary topics were covered by at least one resource. The median rMETRIQ score was 11 of 21 (interquartile range = 8-14). Calculus of urinary tract was most prominently featured with 60 posts. Thirty-four posts (10% of full-text screened FOAM articles) covering 12 core topics were identified as high quality (rMETRIQ ≥ 16). CONCLUSIONS We demonstrated the feasibility of systematically identifying and curating FOAM resources for a specific EM topic and identified an overrepresentation of some subtopics. This curated list of resources may guide trainees, teacher recommendations, and resource producers. Further entries in the series will address other topics relevant to EM.
Collapse
Affiliation(s)
- Andrew Grock
- Division of Emergency MedicineGreater Los Angeles VA Healthcare System, and the Department of Emergency MedicineDavid Geffen School of MedicineUCLALos AngelesCA
| | - Anuja Bhalerao
- School of MedicineUniversity of TorontoTorontoOntarioCanada
| | - Teresa M. Chan
- Department of MedicineDivision of Emergency MedicineMcMaster University, and the McMaster Education Research, Innovation, and Theory ProgramHamiltonOntarioCanada
| | - Brent Thoma
- Department of Emergency MedicineUniversity of SaskatchewanSaskatoonSaskatchewanCanada
| | - Annie B. Wescott
- Galter Health Sciences Library & Learning CenterFeinberg School of MedicineChicagoIL
| | - N. Seth Trueger
- Department of Emergency MedicineNorthwestern UniversityChicagoIL
| |
Collapse
|
32
|
Colmers‐Gray IN, Krishnan K, Chan TM, Seth Trueger N, Paddock M, Grock A, Zaver F, Thoma B. The Revised METRIQ Score: A Quality Evaluation Tool for Online Educational Resources. AEM EDUCATION AND TRAINING 2019; 3:387-392. [PMID: 31637356 PMCID: PMC6795360 DOI: 10.1002/aet2.10376] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/26/2019] [Revised: 06/08/2019] [Accepted: 06/17/2019] [Indexed: 05/12/2023]
Abstract
BACKGROUND With the rapid proliferation of online medical education resources, quality evaluation is increasingly critical. The Medical Education Translational Resources: Impact and Quality (METRIQ) study evaluated the METRIQ-8 quality assessment instrument for blogs and collected feedback to improve it. METHODS As part of the larger METRIQ study, participants rated the quality of five blog posts on clinical emergency medicine topics using the eight-item METRIQ-8 score. Next, participants used a 7-point Likert scale and free-text comments to evaluate the METRIQ-8 score on ease of use, clarity of items, and likelihood of recommending it to others. Descriptive statistics were calculated and comments were thematically analyzed to guide the development of a revised METRIQ (rMETRIQ) score. RESULTS A total of 309 emergency medicine attendings, residents, and medical students completed the survey. The majority of participants felt the METRIQ-8 score was easy to use (mean ± SD = 2.7 ± 1.1 out of 7, with 1 indicating strong agreement) and would recommend it to others (2.7 ± 1.3 out of 7, with 1 indicating strong agreement). The thematic analysis suggested clarifying ambiguous questions, shortening the 7-point scale, specifying scoring anchors for the questions, eliminating the "unsure" option, and grouping-related questions. This analysis guided changes that resulted in the rMETRIQ score. CONCLUSION Feedback on the METRIQ-8 score contributed to the development of the rMETRIQ score, which has improved clarity and usability. Further validity evidence on the rMETRIQ score is required.
Collapse
Affiliation(s)
| | - Keeth Krishnan
- Department of Family MedicineMcMaster UniversityHamiltonOntarioCanada
| | - Teresa M. Chan
- Division of Emergency MedicineDepartment of MedicineMcMaster UniversityHamiltonOntarioCanada
| | - N. Seth Trueger
- Department of Emergency MedicineNorthwestern UniversityChicagoIL
| | - Michael Paddock
- Department of Emergency MedicineUniversity of MinnesotaMinneapolisMN
| | - Andrew Grock
- USC and LAC Departments of Emergency MedicineDavid Geffen School of Medicine at UCLALos AngelesCA
| | - Fareen Zaver
- Department of Emergency MedicineUniversity of CalgaryCalgaryAlbertaCanada
| | - Brent Thoma
- Department of Emergency MedicineUniversity of SaskatchewanSaskatoonSaskatchewanCanada
| |
Collapse
|
33
|
Khadpe J, Morley EJ, Rezaie SR, Grock A. Academic Life in Emergency Medicine (ALiEM) Blog and Podcast Watch: Gastrointestinal Emergencies. Cureus 2019; 11:e5545. [PMID: 31673482 PMCID: PMC6819072 DOI: 10.7759/cureus.5545] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/02/2019] [Accepted: 09/01/2019] [Indexed: 01/25/2023] Open
Abstract
The Academic Life in Emergency Medicine (ALiEM) Approved Instructional Resources (AIR) Series and Approved Instructional Resources - Professional (AIR-Pro) Series were created in 2014 and 2015, respectively, in response to the growing need to curate online educational content as well as create a nationally available curriculum that meets individualized interactive instruction criteria for emergency medicine (EM) trainees. These two online series identify high-quality educational blog and podcast content using an expert-based approach. We summarize the accredited posts on gastrointestinal emergencies that met our a priori determined quality criteria per evaluation by eight experienced faculty educators in EM.
Collapse
Affiliation(s)
- Jay Khadpe
- Emergency Medicine, University of Florida College of Medicine, Jacksonville, USA
| | - Eric J Morley
- Emergency Medicine, Stony Brook Medicine, New York, USA
| | - Salim R Rezaie
- Emergency Medicine, Methodist Hospital, San Antonio, USA
| | - Andrew Grock
- Emergency Medicine, University of California, Los Angeles, USA
| |
Collapse
|
34
|
Chan TM, Kuehl DR. On Lampposts, Sneetches, and Stars: A Call to Go Beyond Bibliometrics for Determining Academic Value. Acad Emerg Med 2019; 26:688-694. [PMID: 30706552 DOI: 10.1111/acem.13707] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
Affiliation(s)
- Teresa M. Chan
- Department of Medicine; Division of Emergency Medicine; McMaster University; Hamilton Ontario Canada
| | - Damon R. Kuehl
- Department of Emergency Medicine; Virginia Tech Carilion School of Medicine; Roanoke VA
| |
Collapse
|
35
|
Zhi A, Husain A. Free open access medical education applications: a critical appraisal of techniques for quality assessment and content discovery. Clin Exp Emerg Med 2019; 6:93-95. [PMID: 30944295 PMCID: PMC6453695 DOI: 10.15441/ceem.17.292] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/22/2017] [Accepted: 01/17/2018] [Indexed: 11/29/2022] Open
Affiliation(s)
- Allen Zhi
- Department of Emergency Medicine, Northwell Health Staten Island University Hospital, Staten Island, NY, USA
| | - Abbas Husain
- Department of Emergency Medicine, Northwell Health Staten Island University Hospital, Staten Island, NY, USA
| |
Collapse
|
36
|
Dubosh NM, Jordan J, Yarris LM, Ullman E, Kornegay J, Runde D, Juve AM, Fisher J. Critical Appraisal of Emergency Medicine Educational Research: The Best Publications of 2016. AEM EDUCATION AND TRAINING 2019; 3:58-73. [PMID: 30680348 PMCID: PMC6339548 DOI: 10.1002/aet2.10203] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/15/2018] [Revised: 09/27/2018] [Accepted: 10/02/2018] [Indexed: 05/05/2023]
Abstract
OBJECTIVES The objectives were to critically appraise the emergency medicine (EM) medical education literature published in 2016 and review the highest-quality quantitative and qualitative studies. METHODS A search of the English language literature in 2016 querying MEDLINE, Scopus, Education Resources Information Center (ERIC), and PsychInfo identified 510 papers related to medical education in EM. Two reviewers independently screened all of the publications using previously established exclusion criteria. The 25 top-scoring quantitative studies based on methodology and all six qualitative studies were scored by all reviewers using selected scoring criteria that have been adapted from previous installments. The top-scoring articles were highlighted and trends in medical education research were described. RESULTS Seventy-five manuscripts met inclusion criteria and were scored. Eleven quantitative and one qualitative papers were the highest scoring and are summarized in this article. CONCLUSION This annual critical appraisal series highlights the best EM education research articles published in 2016.
Collapse
Affiliation(s)
- Nicole M. Dubosh
- Beth Israel Deaconess Medical Center and Harvard Medical SchoolBostonMA
| | - Jaime Jordan
- University of California Los Angeles School of MedicineTorranceCA
| | | | - Edward Ullman
- Beth Israel Deaconess Medical Center and Harvard Medical SchoolBostonMA
| | | | | | | | - Jonathan Fisher
- University of Arizona College of Medicine PhoenixMaricopa Medical CenterPhoenixAZ
| |
Collapse
|
37
|
Min AA, Jordan J, Swaminathan A, Hennings J, Grock A. Academic Life in Emergency Medicine (ALiEM) Blog and Podcast Watch: Renal and Genitourinary Emergencies. Cureus 2018; 10:e3756. [PMID: 30820376 PMCID: PMC6388816 DOI: 10.7759/cureus.3756] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022] Open
Abstract
The Academic Life in Emergency Medicine (ALiEM) Approved Instructional Resources (AIR) series and Approved Instruction Resources Professional (AIR-Pro) series were created in 2014 and 2015, respectively, to address the need for the curation of online educational content as well as a nationally available curriculum that meets individualized interactive instruction criteria. These two programs identify high-quality educational blog and podcast content using an expert-based approach. The AIR series is a continuously building curriculum originally based on the Council of Emergency Medicine Directors (CORD) testing schedule. Using the ALiEM AIR scoring instrument, 49 blog posts and podcasts relevant to renal and genitourinary emergencies published within the previous 12 months were evaluated by eight attending physicians. We summarize the 13 posts that met our a priori determined quality criteria per evaluation by the reviewers. The ALiEM Blog and Podcast Watch series identifies high-quality educational blogs and podcasts for emergency medicine clinicians through its expert panel, using a validated scoring instrument. While this article focuses on renal and genitourinary emergencies, additional AIR modules address other topics in emergency medicine. The AIR and AIR-Pro series provide post-publication accreditation and curation of recent online content to identify and recommend high-quality educational social media content for the EM clinician.
Collapse
Affiliation(s)
- Alice A Min
- Emergency Medicine, University of Arizona College of Medicine, Tucson, USA
| | - Jaime Jordan
- Emergency Medicine, University of California, Los Angeles, USA
| | | | - Jacob Hennings
- Emergency Medicine, University of Tennessee Health Science Center College of Medicine, Chattanooga, USA
| | - Andrew Grock
- Emergency Medicine, University of California, Los Angeles, USA
| |
Collapse
|
38
|
Chary M, Parikh S, Manini AF, Boyer EW, Radeos M. A Review of Natural Language Processing in Medical Education. West J Emerg Med 2018; 20:78-86. [PMID: 30643605 PMCID: PMC6324711 DOI: 10.5811/westjem.2018.11.39725] [Citation(s) in RCA: 31] [Impact Index Per Article: 5.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/12/2018] [Revised: 11/07/2018] [Accepted: 11/16/2018] [Indexed: 11/11/2022] Open
Abstract
Natural language processing (NLP) aims to program machines to interpret human language as humans do. It could quantify aspects of medical education that were previously amenable only to qualitative methods. The application of NLP to medical education has been accelerating over the past several years. This article has three aims. First, we introduce the reader to NLP. Second, we discuss the potential of NLP to help integrate FOAM (Free Open Access Medical Education) resources with more traditional curricular elements. Finally, we present the results of a systematic review. We identified 30 articles indexed by PubMed as relating to medical education and NLP, 14 of which were of sufficient quality to include in this review. We close by discussing potential future work using NLP to advance the field of medical education in emergency medicine.
Collapse
Affiliation(s)
- Michael Chary
- New York-Presbyterian/Queens, Department of Emergency Medicine, Flushing, New York.,Boston Children's Hospital, Harvard Medical Toxicology, Boston, Massachusetts.,Brigham and Women's Hospital, Department of Emergency Medicine, Boston, Massachusetts
| | - Saumil Parikh
- New York-Presbyterian/Queens, Department of Emergency Medicine, Flushing, New York
| | - Alex F Manini
- Icahn School of Medicine at Mount Sinai, Elmhurst Hospital Center, Department of Emergency Medicine, New York, New York
| | - Edward W Boyer
- Brigham and Women's Hospital, Department of Emergency Medicine, Boston, Massachusetts
| | - Michael Radeos
- Icahn School of Medicine at Mount Sinai, Elmhurst Hospital Center, Department of Emergency Medicine, New York, New York.,Coney Island Hospital, Department of Emergency Medicine, Brooklyn, New York
| |
Collapse
|
39
|
Grock A, Wheaton N, Roppolo L, Gaafary C. Academic Life in Emergency Medicine Blog and Podcast Watch: Toxicologic Emergencies. Cureus 2018; 10:e3687. [PMID: 30761239 PMCID: PMC6368425 DOI: 10.7759/cureus.3687] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022] Open
Abstract
The Academic Life in Emergency Medicine (ALiEM) Approved Instructional Resources (AIR) Series was created in 2014 to address a lack of both curation of online educational content and a nationally available curriculum that meets individualized interactive instruction. Using an expert-based, crowdsourced approach, the AIR series identifies trustworthy, high-quality, educational blog and podcast content. Here, we summarize the content rated as high quality per our a priori criteria as evaluated by eight attending physicians.
Collapse
Affiliation(s)
- Andrew Grock
- Emergency Medicine, University of California, Los Angeles, USA
| | - Natasha Wheaton
- Emergency Medicine, University of California, Los Angeles, USA
| | - Lynn Roppolo
- Emergency Medicine, University of Texas Southwestern, Dallas, USA
| | - Chris Gaafary
- Emergency Medicine, University of South Carolina College of Medicine, Greenville, USA
| |
Collapse
|
40
|
Young TP, Guptill M, Thomas T, Mellick L. Effective Educational Videos in Emergency Medicine. AEM EDUCATION AND TRAINING 2018; 2:S17-S24. [PMID: 30607375 PMCID: PMC6304276 DOI: 10.1002/aet2.10210] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/31/2018] [Revised: 10/28/2018] [Accepted: 10/29/2018] [Indexed: 06/01/2023]
Abstract
Video is a popular format for teaching and learning online. Emergency medicine (EM) has been a leader in online learning and EM educators may wish to use video to teach. The creation of online video content is fraught with pitfalls that may make videos less effective. We review notable theory and evidence regarding effective use of video for education in EM with international considerations.
Collapse
Affiliation(s)
- Timothy P. Young
- Department of Emergency MedicineLoma Linda University Medical Center and Children's HospitalMedical Simulation CenterLoma Linda University Medical SchoolLoma LindaCA
| | - Mindi Guptill
- Department of Emergency MedicineLoma Linda University Medical Center and Children's HospitalMedical Simulation CenterLoma Linda University Medical SchoolLoma LindaCA
| | - Tammi Thomas
- Department of Emergency MedicineLoma Linda University Medical Center and Children's HospitalMedical Simulation CenterLoma Linda University Medical SchoolLoma LindaCA
| | - Larry Mellick
- Department of Emergency MedicineUniversity of South AlabamaMobileAL
| |
Collapse
|
41
|
Thoma B, Chan TM, Kapur P, Sifford D, Siemens M, Paddock M, Ankel F, Grock A, Lin M. The Social Media Index as an Indicator of Quality for Emergency Medicine Blogs: A METRIQ Study. Ann Emerg Med 2018; 72:696-702. [PMID: 29980461 DOI: 10.1016/j.annemergmed.2018.05.003] [Citation(s) in RCA: 34] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/09/2018] [Revised: 03/31/2018] [Accepted: 05/01/2018] [Indexed: 11/26/2022]
Abstract
STUDY OBJECTIVE Online educational resources such as blogs are increasingly used for education by emergency medicine clinicians. The Social Media Index was developed to quantify their relative impact. The Medical Education Translational Resources: Indicators of Quality (METRIQ) study was conducted in part to determine the association between the Social Media Index score and quality as measured by gestalt and previously derived quality instruments. METHODS Ten blogs were randomly selected from a list of emergency medicine and critical care Web sites. The 2 most recent clinically oriented blog posts published on these blogs were evaluated with gestalt, the Academic Life in Emergency Medicine Approved Instructional Resources (ALiEM AIR) score, and the METRIQ-8 score. Volunteer raters (including medical students, emergency medicine residents, and emergency medicine attending physicians) were identified with a multimodal recruitment methodology. The Social Media Index was calculated in February 2016, November 2016, April 2017, and December 2017. Pearson's correlations were calculated between the Social Media Index and the average rater gestalt, ALiEM AIR score, and METRIQ-8 score. RESULTS A total of 309 of 330 raters completed all ratings (93.6%). The Social Media Index correlated moderately to strongly with the mean rater gestalt ratings (range 0.69 to 0.76) and moderately with the mean rater ALiEM AIR score (range 0.55 to 0.61) and METRIQ-8 score (range 0.53 to 0.57) during the month of the blog post's selection and for 2 years after. CONCLUSION The Social Media Index's correlation with multiple quality evaluation instruments over time supports the hypothesis that it is associated with overall Web site quality. It can play a role in guiding individuals to high-quality resources that can be reviewed with critical appraisal techniques.
Collapse
Affiliation(s)
- Brent Thoma
- Department of Emergency Medicine, University of Saskatchewan, Saskatoon, Saskatchewan, Canada; Health Professions Education, Massachusetts General Hospital Institute, Boston, MA.
| | - Teresa M Chan
- Division of Emergency Medicine, Department of Medicine, McMaster University, Hamilton, Ontario, Canada
| | - Puneet Kapur
- Department of Emergency Medicine, University of Saskatchewan, Saskatoon, Saskatchewan, Canada
| | | | - Marshall Siemens
- Department of Emergency Medicine, University of Saskatchewan, Saskatoon, Saskatchewan, Canada
| | - Michael Paddock
- Department of Emergency Medicine, Regions Hospital, HealthPartners Institute, Saint Paul, MN, and the Department of Emergency Medicine, University of Minnesota, Saint Paul, MN
| | - Felix Ankel
- Department of Emergency Medicine, Regions Hospital, HealthPartners Institute, Saint Paul, MN, and the Department of Emergency Medicine, University of Minnesota, Saint Paul, MN; HealthPartners Institute, Saint Paul, MN
| | - Andy Grock
- USC+LAC Department of Emergency Medicine and David Geffen School of Medicine at UCLA, Los Angeles, CA
| | - Michelle Lin
- Department of Emergency Medicine, University of California-San Francisco, San Francisco, CA
| | | |
Collapse
|
42
|
Thoma B, Sebok-Syer SS, Colmers-Gray I, Sherbino J, Ankel F, Trueger NS, Grock A, Siemens M, Paddock M, Purdy E, Kenneth Milne W, Chan TM. Quality Evaluation Scores are no more Reliable than Gestalt in Evaluating the Quality of Emergency Medicine Blogs: A METRIQ Study. TEACHING AND LEARNING IN MEDICINE 2018; 30:294-302. [PMID: 29381099 DOI: 10.1080/10401334.2017.1414609] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/07/2023]
Abstract
UNLABELLED Construct: We investigated the quality of emergency medicine (EM) blogs as educational resources. PURPOSE Online medical education resources such as blogs are increasingly used by EM trainees and clinicians. However, quality evaluations of these resources using gestalt are unreliable. We investigated the reliability of two previously derived quality evaluation instruments for blogs. APPROACH Sixty English-language EM websites that published clinically oriented blog posts between January 1 and February 24, 2016, were identified. A random number generator selected 10 websites, and the 2 most recent clinically oriented blog posts from each site were evaluated using gestalt, the Academic Life in Emergency Medicine (ALiEM) Approved Instructional Resources (AIR) score, and the Medical Education Translational Resources: Impact and Quality (METRIQ-8) score, by a sample of medical students, EM residents, and EM attendings. Each rater evaluated all 20 blog posts with gestalt and 15 of the 20 blog posts with the ALiEM AIR and METRIQ-8 scores. Pearson's correlations were calculated between the average scores for each metric. Single-measure intraclass correlation coefficients (ICCs) evaluated the reliability of each instrument. RESULTS Our study included 121 medical students, 88 EM residents, and 100 EM attendings who completed ratings. The average gestalt rating of each blog post correlated strongly with the average scores for ALiEM AIR (r = .94) and METRIQ-8 (r = .91). Single-measure ICCs were fair for gestalt (0.37, IQR 0.25-0.56), ALiEM AIR (0.41, IQR 0.29-0.60) and METRIQ-8 (0.40, IQR 0.28-0.59). CONCLUSION The average scores of each blog post correlated strongly with gestalt ratings. However, neither ALiEM AIR nor METRIQ-8 showed higher reliability than gestalt. Improved reliability may be possible through rater training and instrument refinement.
Collapse
Affiliation(s)
- Brent Thoma
- a Department of Emergency Medicine , College of Medicine, University of Saskatchewan , Saskatoon , Saskatchewan , Canada
| | - Stefanie S Sebok-Syer
- b Center for Education Research & Innovation, Schulich School of Medicine and Dentistry, Western University , London , Ontario , Canada
| | - Isabelle Colmers-Gray
- c Department of Emergency Medicine University of Alberta , Edmonton , Alberta , Canada
| | - Jonathan Sherbino
- d Department of Emergency Medicine , McMaster University , Hamilton , Ontario , Canada
| | - Felix Ankel
- e Department of Health Professions Education at HealthPartners Institute , Bloomington , Minnesota , USA
| | - N Seth Trueger
- f Department of Emergency Medicine , Northwestern University , Chicago , Illinois , USA
| | - Andrew Grock
- g Department of Emergency Medicine , University of California Los Angeles , Los Angeles , California , USA
| | - Marshall Siemens
- h College of Medicine, University of Saskatchewan , Saskatoon , Saskatchewan , Canada
| | - Michael Paddock
- i Department of Emergency Medicine , University of Minnesota , Minneapolis , Minnesota , USA
| | - Eve Purdy
- j Department of Emergency Medicine , Queen's University, Kingston , Ontario , Canada
| | | | - Teresa M Chan
- d Department of Emergency Medicine , McMaster University , Hamilton , Ontario , Canada
| |
Collapse
|
43
|
Azim A, Beck‐Esmay J, Chan TM. Editorial Processes in Free Open Access Medical Educational (FOAM) Resources. AEM EDUCATION AND TRAINING 2018; 2:204-212. [PMID: 30051090 PMCID: PMC6050060 DOI: 10.1002/aet2.10097] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/11/2018] [Revised: 02/22/2018] [Accepted: 03/05/2018] [Indexed: 05/21/2023]
Abstract
BACKGROUND Much of the skepticism toward online educational resources (OERs) in emergency medicine (EM) stems from the low barrier to publishing and a perceived lack of editorial rigor. Learners and educators have demonstrated unreliable gestalt ratings of OERs, suggesting a lack of capacity to consistently appraise these resources. The development of tools to guide clinicians and learners in the selection and use of blogs and podcasts is a growing area of interest. Disclosure of editorial process was identified in previous studies as an important quality indicator for OERs. However, little is known about editorial process in online EM resources and whether it can be reliably integrated into a critical appraisal tool. METHODS Two reviewers assessed 100 top EM and critical care OERs for mention and description of editorial process and academic and nonacademic affiliations. Ninety-two sites were accessible for review. All sites were also contacted to attempt clarification of their editorial process. Inter-rater reliability for mention and description of editorial process was evaluated using Cohen's kappa, and the relationship between academic affiliation and disclosure of editorial process was assessed by odds ratio (OR). RESULTS Eleven sites mentioned an editorial process, and 10 of these sites included a description. Five of the seven sites that responded to contact also described an editorial process. Inter-rater agreement was excellent for mention (κ = 0.90) and description (κ = 1.00) of editorial process. Eighteen sites listed academic affiliations and 21 sites had nonacademic affiliations. A greater proportion of sites with academic affiliations disclosed their editorial process compared to sites without academic affiliations (OR = 5.3, 95% confidence interval [CI] = 1.3-21.0; difference in proportions of 0.40, 95% CI = 11.6-60.8). CONCLUSIONS Although transparency is lacking, editorial processes exist among OERs. Inter-rater reliability for disclosure of editorial process is excellent, supporting its use within critical appraisal tools.
Collapse
Affiliation(s)
- Arden Azim
- Michael G. DeGroote School of MedicineMcMaster UniversityHamiltonOntarioCanada
| | | | - Teresa M. Chan
- Division of Emergency MedicineDepartment of MedicineMcMaster UniversityHamiltonOntarioCanada
| |
Collapse
|
44
|
Min AA, Morley EJ, Rezaie SR, Fox SM, Grock A. Academic Life in Emergency Medicine Blog and Podcast Watch: Respiratory Emergencies. Cureus 2018; 10:e2812. [PMID: 30116685 PMCID: PMC6092192 DOI: 10.7759/cureus.2812] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022] Open
Abstract
The Academic Life in Emergency Medicine (ALiEM) Approved Instructional Resources (AIR) Series and Approved Instruction Resources Professional (AIR-Pro) Series were created in 2014 and 2015, respectively, to address the need for curation of online educational content as well as a nationally available curriculum that meets individualized interactive instruction criteria. These two programs identify high-quality educational blog and podcast content using an expert-based approach. We summarize the accredited posts on respiratory emergencies that met our a priori determined quality criteria per evaluation by eight experienced faculty educators in emergency medicine.
Collapse
Affiliation(s)
- Alice A Min
- Emergency Medicine, University of Arizona College of Medicine, Tucson, USA
| | - Eric J Morley
- Emergency Medicine, Stony Brook Medicine, New York, USA
| | - Salim R Rezaie
- Emergency Medicine/internal Medicine, University of Texas, San Antonio, USA
| | - Sean M Fox
- Emergency Medicine, Carolinas Medical Center, Charlotte, USA
| | - Andrew Grock
- Emergency Medicine, University of California Los Angeles, Los Angeles, USA
| |
Collapse
|
45
|
Gates A, Featherstone R, Shave K, Scott SD, Hartling L. Dissemination of evidence in paediatric emergency medicine: a quantitative descriptive evaluation of a 16-week social media promotion. BMJ Open 2018; 8:e022298. [PMID: 29880576 PMCID: PMC6009559 DOI: 10.1136/bmjopen-2018-022298] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/03/2022] Open
Abstract
OBJECTIVES TRanslating Emergency Knowledge for Kids (TREKK) and Cochrane Child Health collaborate to develop knowledge products on paediatric emergency medicine topics. Via a targeted social media promotion, we aimed to increase user interaction with the TREKK and Cochrane Child Health Twitter accounts and the uptake of TREKK Bottom Line Recommendations (BLRs) and Cochrane systematic reviews (SRs). DESIGN Quantitative descriptive evaluation. SETTING We undertook this study and collected data via the internet. PARTICIPANTS Our target users included online healthcare providers and health consumers. INTERVENTION For 16 weeks, we used Twitter accounts (@TREKKca and @Cochrane_Child) and the Cochrane Child Health blog to promote 6 TREKK BLRs and 16 related Cochrane SRs. We published 1 blog post and 98 image-based tweets per week. PRIMARY AND SECONDARY OUTCOME MEASURES The primary outcome was user interaction with @TREKKca and @Cochrane_Child. Secondary outcomes were visits to TREKK's website and the Cochrane Child Health blog, clicks to and views of the TREKK BLRs, and Altmetric scores and downloads of Cochrane SRs. RESULTS Followers to @TREKKca and @Cochrane_Child increased by 24% and 15%, respectively. Monthly users of TREKK's website increased by 29%. Clicks to the TREKK BLRs increased by 22%. The BLRs accrued 59% more views compared with the baseline period. The 16 blog posts accrued 28% more views compared with the 8 previous months when no new posts were published. The Altmetric scores for the Cochrane SRs increased by ≥10 points each. The mean number of full text downloads for the promotion period was higher for nine and lower for seven SRs compared with the 16-week average for the previous year (mean difference (SD), +4.0 (22.0%)). CONCLUSIONS There was increased traffic to TREKK knowledge products and Cochrane SRs during the social media promotion. Quantitative evidence supports blogging and tweeting as dissemination strategies for evidence-based knowledge products.
Collapse
Affiliation(s)
- Allison Gates
- Alberta Research Centre for Health Evidence (ARCHE), Department of Pediatrics, University of Alberta, Edmonton, Alberta, Canada
| | - Robin Featherstone
- Alberta Research Centre for Health Evidence (ARCHE), Department of Pediatrics, University of Alberta, Edmonton, Alberta, Canada
| | - Kassi Shave
- Alberta Research Centre for Health Evidence (ARCHE), Department of Pediatrics, University of Alberta, Edmonton, Alberta, Canada
| | - Shannon D Scott
- Faculty of Nursing, University of Alberta, Edmonton, Alberta, Canada
| | - Lisa Hartling
- Alberta Research Centre for Health Evidence (ARCHE), Department of Pediatrics, University of Alberta, Edmonton, Alberta, Canada
- Cochrane Child Health, Department of Pediatrics, University of Alberta, Edmonton, Alberta, Canada
| |
Collapse
|
46
|
Bucher J, Donovan C, McCoy J. EMS providers do not use FOAM for education. Int J Emerg Med 2018; 11:27. [PMID: 29797111 PMCID: PMC5968013 DOI: 10.1186/s12245-018-0189-4] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/30/2018] [Accepted: 05/15/2018] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND Free open access to medical education (FOAM, #FOAM) is the free availability of educational materials on various medicine topics. We hope to evaluate the use of social media and FOAM by emergency medical services (EMS) providers. METHODS We designed an online survey distributed to EMS providers with questions about demographics and social media/FOAM use by providers. The survey was sent to the American College of Emergency Physicians (ACEP) EMS Listserv of medical directors and was asked to be distributed to their respective agencies. The survey was designed to inquire about the providers' knowledge of FOAM and social media and their use of the above for EMS education. RESULTS There were 169 respondents out of a total of 523 providers yielding a response rate of 32.3%. Fifty-three percent of respondents are paramedics, 37% are EMT-Basic trained, and the remainder (16%) were "other." The minority (20%) of respondents had heard of FOAM. However, 54% of respondents had heard of "free medical education online" regarding pertinent topics. Of the total respondents who used social media for education, 31% used Facebook and 23% used blogs and podcasts as resources for online education. Only 4% of respondents stated they produced FOAM content. Seventy-six percent of respondents said they were "interested" or "very interested" in using FOAM for medical education. If FOAM provided continuing medical education (CME), 83% of respondents would be interested in using it. CONCLUSION Social media is not used frequently by EMS providers for the purposes of FOAM. There is interest within EMS providers to use FOAM for education, even if CME was not provided. FOAM can provide a novel area of education for EMS.
Collapse
Affiliation(s)
- Joshua Bucher
- Department of Emergency Medicine, Rutgers - Robert Wood Johnson Medical School, 1 RWJ Place, MEB 104, New Brunswick, NJ, 0890, USA.
| | - Colleen Donovan
- Department of Emergency Medicine, Rutgers - Robert Wood Johnson Medical School, 1 RWJ Place, MEB 104, New Brunswick, NJ, 0890, USA
| | - Jonathan McCoy
- Department of Emergency Medicine, Rutgers - Robert Wood Johnson Medical School, 1 RWJ Place, MEB 104, New Brunswick, NJ, 0890, USA
| |
Collapse
|
47
|
Franzen D, Cooney R, Chan T, Brown M, Diercks DB. Scholarship by the Clinician-Educator in Emergency Medicine. AEM EDUCATION AND TRAINING 2018; 2:115-120. [PMID: 30051078 PMCID: PMC6001503 DOI: 10.1002/aet2.10084] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/01/2017] [Revised: 12/28/2017] [Accepted: 01/23/2018] [Indexed: 05/25/2023]
Abstract
Emergency medicine (EM) continues to grow as an academic specialty. Like most specialties, a large number of academic emergency physicians are focused on education of our graduate student learners. For promotion, clinician-educators (CEs) are required to produce scholarly work and disseminate knowledge. Although promotion requirements may vary by institution, scholarly work is a consistent requirement. Due to the clinical constraints of working in the emergency department, the unique interactions emergency physicians have with their learners, and early adoption of alternative teaching methods, EM CEs' scholarly work may not be adequately described in a traditional curriculum vitae. Using a rubric of established domains around the academic work of CEs, this article describes some of the ways EM educators address these domains. The aim of the article is to provide a guide for academic department leadership, CEs, and promotion committees about the unique ways EM has addressed the work of the CE.
Collapse
Affiliation(s)
- Douglas Franzen
- Division of Emergency MedicineUniversity of WashingtonSeattleWA
| | - Robert Cooney
- Department of Emergency MedicineGeisinger Medical CenterDanvillePA
| | - Teresa Chan
- Division of Emergency MedicineDepartment of MedicineMcMaster UniversityHamiltonOntarioCanada
| | - Michael Brown
- Department of Emergency MedicineMichigan State University College of Human MedicineGrand RapidsMI
| | - Deborah B. Diercks
- Department of Emergency MedicineUniversity of Texas SouthwesternDallasTX
| |
Collapse
|
48
|
Roppolo L, Gaafary C, Khadpe J, Shah K, Grock A. Academic Life in Emergency Medicine (ALiEM) Blog and Podcast Watch: Infectious Disease Emergencies. Cureus 2018; 10:e2345. [PMID: 29796357 PMCID: PMC5959306 DOI: 10.7759/cureus.2345] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022] Open
Abstract
To address the needs for curation of online educational content as well as the development of a nationally available curriculum that meets individualized interactive instruction, the Academic Life in Emergency Medicine (ALiEM) Approved Instructional Resources (AIR) Series and AIR-Pro Series were created in 2014 and 2015, respectively. Using an expert-based, crowd-sourced approach, these two programs identify trustworthy, high-quality, educational blog, and podcast content. Here, we summarize the accredited posts that met our a priori determined quality criteria and evaluated by eight attending physicians.
Collapse
Affiliation(s)
- Lynn Roppolo
- Department of Emergency Medicine, University of Texas Southwestern, Dallas, Texas
| | - Chris Gaafary
- University of South Carolina College of Medicine Greenville, University of South Carolina College of Medicine Greenville / Greenville Health System, Greenville, Sc
| | - Jay Khadpe
- Department of Emergency Medicine, University of Florida College of Medicine, Jacksonville, Florida
| | - Kaushal Shah
- Department of Emergency Medicine, Icahn School of Medicine at Mount Sinai Medical Center, New York, Ny
| | | |
Collapse
|
49
|
|
50
|
Quinn A, Chan TM, Sampson C, Grossman C, Butts C, Casey J, Caretta-Weyer H, Gottlieb M. Curated Collections for Educators: Five Key Papers on Evaluating Digital Scholarship. Cureus 2018; 10:e2021. [PMID: 29531874 PMCID: PMC5837232 DOI: 10.7759/cureus.2021] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/11/2017] [Accepted: 01/03/2018] [Indexed: 11/17/2022] Open
Abstract
Traditionally, scholarship that was recognized for promotion and tenure consisted of clinical research, bench research, and grant funding. Recent trends have allowed for differing approaches to scholarship, including digital publication. As increasing numbers of trainees and faculty turn to online educational resources, it is imperative to critically evaluate these resources. This article summarizes five key papers that address the appraisal of digital scholarship and describes their relevance to junior clinician educators and faculty developers. In May 2017, the Academic Life in Emergency Medicine Faculty Incubator program focused on the topic of digital scholarship, providing and discussing papers relevant to the topic. We augmented this list of papers with further suggestions by guest experts and by an open call via Twitter for other important papers. Through this process, we created a list of 38 papers in total on the topic of evaluating digital scholarship. In order to determine which of these papers best describe how to evaluate digital scholarship, the authorship group assessed the papers using a modified Delphi approach to build consensus. In this paper we present the five most highly rated papers from our process about evaluating digital scholarship. We summarize each paper and discuss its specific relevance to junior faculty members and to faculty developers. These papers provide a framework for assessing the quality of digital scholarship, so that junior faculty can recommend high-quality educational resources to their trainees. These papers help guide educators on how to produce high quality digital scholarship and maximize recognition and credit in respect to receiving promotion and tenure.
Collapse
Affiliation(s)
- Antonia Quinn
- Department of Emergency Medicine, SUNY Downstate College of Medicine
| | - Teresa M Chan
- Faculty of Health Sciences, Department of Medicine, Division of Emergency Medicine, McMaster University
| | | | - Catherine Grossman
- Pulmonary and Critical Care Medicine, Virginia Commonwealth University Health Systems
| | - Christine Butts
- Section of Emergency Medicine, Louisiana State University Health Sciences
| | - John Casey
- Department of Emergency Medicine, Ohiohealth Doctors Hospital
| | | | - Michael Gottlieb
- Department of Emergency Medicine, Rush University Medical Center
| |
Collapse
|