Using the Electronic Health Record to Identify Educational Gaps for Internal Medicine Interns

Using the electronic health record to provide audit and feedback in medical student clerkships

Objectives

This article focuses on the role of the electronic health record (EHR) to generate meaningful formative feedback for medical students in the clinical setting. Despite the scores of clinical data housed within the EHR, medical educators have only just begun to tap into this data to enhance student learning. Literature to-date has focused almost exclusively on resident education.

Materials and Methods

Development of EHR auto-logging and triggered notifications are discussed as specific use cases in providing enhanced feedback for medical students.

Results

By incorporating predictive and prescriptive analytics into the EHR, there is an opportunity to create powerful educational tools which may also support general clinical activity.

Discussion

This article explores the possibilities of EHR as an educational resource. This serves as a call to action for educators and technology developers to work together on creating health record user-centric tools, acknowledging the ongoing work done to improve student-level attribution to patients.

Conclusion

EHR analytics and tools present a novel approach to enhancing clinical clerkship education for medical students.

Toward precision medical education: Characterizing individual residents' clinical experiences throughout training

BACKGROUND

Despite the central role of experiential learning in residency training, the actual clinical experiences residents participate in are not well characterized. A better understanding of the type, volume, and variation in residents' clinical experiences is essential to support precision medical education strategies.

OBJECTIVE

We sought to characterize the entirety of the clinical experiences had by individual internal medicine residents throughout their time in training.

METHOD

We evaluated the clinical experiences of medicine residents (n = 51) who completed training at NYU Grossman School of Medicine's Brooklyn campus between 2020 and 2023. Residents' inpatient and outpatient experiences were identified using notes written, orders placed, and care team sign-ins; principal ICD-10 codes for each encounter were converted into medical content categories using a previously described crosswalk tool.

RESULTS

Of 152,426 clinical encounters with available ICD-10 codes, 132,284 were mapped to medical content categories (94.5% capture). Residents' clinical experiences were particularly enriched in infectious and cardiovascular disease; most had very little exposure to allergy, dermatology, oncology, or rheumatology. Some trainees saw twice as many cases in a given content area as did others. There was little concordance between actual frequency of clinical experience and expected content frequency on the ABIM certification exam.

CONCLUSIONS

Individual residents' clinical experiences in training vary widely, both in number and in type. Characterizing these experiences paves the way for exploration of the relationships between clinical exposure and educational outcomes, and for the implementation of precision education strategies that could fill residents' experiential gaps and complement strengths with targeted educational interventions.

The Hospitalist Movement 25 Years Later

Hospitalists are generalists who specialize in the care of hospitalized patients. In the 25 years since the term hospitalist was coined, the field of hospital medicine has grown exponentially and established a substantial footprint in the medical community. There are now more hospitalists than practicing physicians in any other internal medicine subspecialty. Several key forces catalyzed the growth in the field of hospital medicine, including the quality, safety, and value movements; residency duty hour restrictions; the emergence of electronic health records; and the COVID-19 pandemic. Looking ahead, we see new opportunities in the realms of technology and telemedicine, and challenges persist in regard to balancing financial considerations with increasing workload and burnout. Hospitalists must remain nimble and seize emerging opportunities to continue supporting the field's prominence and growth.

Uses of Electronic Health Record Data to Measure the Clinical Learning Environment of Graduate Medical Education Trainees: A Systematic Review

PURPOSE

This study systematically reviews the uses of electronic health record (EHR) data to measure graduate medical education (GME) trainee competencies.

METHOD

In January 2022, the authors conducted a systematic review of original research in MEDLINE from database start to December 31, 2021. The authors searched for articles that used the EHR as their data source and in which the individual GME trainee was the unit of observation and/or unit of analysis. The database query was intentionally broad because an initial survey of pertinent articles identified no unifying Medical Subject Heading terms. Articles were coded and clustered by theme and Accreditation Council for Graduate Medical Education (ACGME) core competency.

RESULTS

The database search yielded 3,540 articles, of which 86 met the study inclusion criteria. Articles clustered into 16 themes, the largest of which were trainee condition experience (17 articles), work patterns (16 articles), and continuity of care (12 articles). Five of the ACGME core competencies were represented (patient care and procedural skills, practice-based learning and improvement, systems-based practice, medical knowledge, and professionalism). In addition, 25 articles assessed the clinical learning environment.

CONCLUSIONS

This review identified 86 articles that used EHR data to measure individual GME trainee competencies, spanning 16 themes and 6 competencies and revealing marked between-trainee variation. The authors propose a digital learning cycle framework that arranges sequentially the uses of EHR data within the cycle of clinical experiential learning central to GME. Three technical components necessary to unlock the potential of EHR data to improve GME are described: measures, attribution, and visualization. Partnerships between GME programs and informatics departments will be pivotal in realizing this opportunity.

Mapping hospital data to characterize residents' educational experiences

BACKGROUND

Experiential learning through patient care is fundamental to graduate medical education. Despite this, the actual content to which trainees are exposed in clinical practice is difficult to quantify and is poorly characterized. There remains an unmet need to define precisely how residents' patient care activities inform their educational experience.  METHODS: Using a recently-described crosswalk tool, we mapped principal ICD-10 discharge diagnosis codes to American Board of Internal Medicine (ABIM) content at four training hospitals of a single Internal Medicine (IM) Residency Program over one academic year to characterize and compare residents' clinical educational experiences. Frequencies of broad content categories and more specific condition categories were compared across sites to profile residents' aggregate inpatient clinical experiences and drive curricular change.

RESULTS

There were 18,604 discharges from inpatient resident teams during the study period. The crosswalk captured > 95% of discharges at each site. Infectious Disease (ranging 17.4 to 39.5% of total discharges) and Cardiovascular Disease (15.8 to 38.2%) represented the most common content categories at each site. Several content areas (Allergy/Immunology, Dermatology, Obstetrics/Gynecology, Ophthalmology, Otolaryngology/Dental Medicine) were notably underrepresented (≤ 1% at each site). There were significant differences in the frequencies of conditions within most content categories, suggesting that residents experience distinct site-specific clinical content during their inpatient training.

CONCLUSIONS

There were substantial differences in the clinical content experienced by our residents across hospital sites, prompting several important programmatic and curricular changes to enrich our residents' hospital-based educational experiences.

Experience and Education in Residency Training: Capturing the Resident Experience by Mapping Clinical Data

PROBLEM

Internal medicine training programs operate under the assumption that the 3-year residency training period is sufficient for trainees to achieve the depth and breadth of clinical experience necessary for independent practice; however, the medical conditions to which residents are exposed in clinical practice are not easily measured. As a result, residents' clinical educational experiences are poorly understood.

APPROACH

A crosswalk tool (a repository of International Classification of Diseases [ICD]-10 codes linked to medical content areas) was developed to query routinely collected inpatient principal diagnosis codes and translate them into an educationally meaningful taxonomy. This tool provides a robust characterization of residents' inpatient clinical experiences.

OUTCOMES

This pilot study has provided proof of principle that the crosswalk tool can effectively map 1 year of resident-attributed diagnosis codes to both the broad content category level (e.g., "cardiovascular disease") and to the more specific condition category level (e.g., "myocardial disease"). The authors uncovered content areas in their training program that are overrepresented and some that are underrepresented relative to material on the American Board of Internal Medicine (ABIM) Certification Exam.

NEXT STEPS

The crosswalk tool introduced here translated residents' patient care activities into discrete, measurable educational content and enabled 1 internal medicine residency program to characterize residents' inpatient educational experience with a high degree of resolution. Leaders of other programs seeking to profile the clinical exposure of their trainees may adopt this strategy. Such clinical content mapping drives innovation in the experiential curriculum, enables comparison across practice sites, and lays the groundwork to test associations between individual clinical exposure and competency-based outcomes, which, in turn, will allow medical educators to draw conclusions regarding how clinical experience reflects clinical competency.

Big Data for Biomedical Education with a Focus on the COVID-19 Era: An Integrative Review of the Literature

Medical education refers to education and training delivered to medical students in order to become a practitioner. In recent decades, medicine has been radically transformed by scientific and computational/digital advances—including the introduction of new information and communication technologies, the discovery of DNA, and the birth of genomics and post-genomics super-specialties (transcriptomics, proteomics, interactomics, and metabolomics/metabonomics, among others)—which contribute to the generation of an unprecedented amount of data, so-called ‘big data’. While these are well-studied in fields such as medical research and methodology, translational medicine, and clinical practice, they remain overlooked and understudied in the field of medical education. For this purpose, we carried out an integrative review of the literature. Twenty-nine studies were retrieved and synthesized in the present review. Included studies were published between 2012 and 2021. Eleven studies were performed in North America: specifically, nine were conducted in the USA and two studies in Canada. Six studies were carried out in Europe: two in France, two in Germany, one in Italy, and one in several European countries. One additional study was conducted in China. Eight papers were commentaries/theoretical or perspective articles, while five were designed as a case study. Five investigations exploited large databases and datasets, while five additional studies were surveys. Two papers employed visual data analytical/data mining techniques. Finally, other two papers were technical papers, describing the development of software, computational tools and/or learning environments/platforms, while two additional studies were literature reviews (one of which being systematic and bibliometric).The following nine sub-topics could be identified: (I) knowledge and awareness of big data among medical students; (II) difficulties and challenges in integrating and implementing big data teaching into the medical syllabus; (III) exploiting big data to review, improve and enhance medical school curriculum; (IV) exploiting big data to monitor the effectiveness of web-based learning environments among medical students; (V) exploiting big data to capture the determinants and signatures of successful academic performance and counteract/prevent drop-out; (VI) exploiting big data to promote equity, inclusion, and diversity; (VII) exploiting big data to enhance integrity and ethics, avoiding plagiarism and duplication rate; (VIII) empowering medical students, improving and enhancing medical practice; and, (IX) exploiting big data in continuous medical education and learning. These sub-themes were subsequently grouped in the following four major themes/topics: namely, (I) big data and medical curricula; (II) big data and medical academic performance; (III) big data and societal/bioethical issues in biomedical education; and (IV) big data and medical career. Despite the increasing importance of big data in biomedicine, current medical curricula and syllabuses appear inadequate to prepare future medical professionals and practitioners that can leverage on big data in their daily clinical practice. Challenges in integrating, incorporating, and implementing big data teaching into medical school need to be overcome to facilitate the training of the next generation of medical professionals. Finally, in the present integrative review, state-of-art and future potential uses of big data in the field of biomedical discussion are envisaged, with a focus on the still ongoing “Coronavirus Disease 2019” (COVID-19) pandemic, which has been acting as a catalyst for innovation and digitalization.

"EMERGing" Electronic Health Record Data Metrics: Insights and Implications for Assessing Residents' Clinical Performance in Emergency Medicine

OBJECTIVES

Competency-based medical education requires that residents are provided with frequent opportunities to demonstrate competence as well as receive effective feedback about their clinical performance. To meet this goal, we investigated how data collected by the electronic health record (EHR) might be used to assess emergency medicine (EM) residents' independent and interdependent clinical performance and how such information could be represented in an EM resident report card.

METHODS

Following constructivist grounded theory methodology, individual semistructured interviews were conducted in 2017 with 10 EM faculty and 11 EM residents across all 5 postgraduate years. In addition to open-ended questions, participants were presented with an emerging list of EM practice metrics and asked to comment on how valuable each would be in assessing resident performance. Additionally, we asked participants the extent to which each metric captured independent or interdependent performance. Data collection and analysis were iterative; analysis employed constant comparative inductive methods.

RESULTS

Participants refined and eliminated metrics as well as added new metrics specific to the assessment of EM residents (e.g., time between signup and first orders). These clinical practice metrics based on data from our EHR database were organized along a spectrum of independent/interdependent performance. We conclude with discussions about the relationship among these metrics, issues in interpretation, and implications of using EHR for assessment purposes.

CONCLUSIONS

Our findings document a systematic approach for developing EM resident assessments, based on EHR data, which incorporate the perspectives of both clinical faculty and residents. Our work has important implications for capturing residents' contributions to clinical performances and distinguishing between independent and interdependent metrics in collaborative workplace-based settings.

Psychiatry Resident Attitudes Toward Practice Habit Data

OBJECTIVE

This descriptive study queries the attitudes of psychiatry residents regarding provision of practice habit data to trainees by residency programs, as required by the Accreditation Council for Graduate Medical Education (ACGME). Identifying trainee perspectives may assist program directors in tailoring practice habit data reporting to better engage residents and to increase resident-reported adherence to the ACGME requirement.

METHODS

Residents at a large, hospital-based adult psychiatry training program completed an anonymous survey of attitudes regarding practice habit data, including perceptions of the residency program's current reporting, preferences toward mechanisms of delivering this data, and perceived relative utility of five hypothetical domains of practice habit data.

RESULTS

Of 61 eligible residents, 52 (85%) completed surveys. Only 29 (56%) recalled receiving prior-year individual practice habit data, and only 10 (19%) recalled receiving team-based data. Seventy-five percent desired more practice habit data. Out of five hypothetical thematic domains for practice habit reporting, residents preferred patient-oriented domains as opposed to process-oriented domains. Resident concerns about dissemination of these data included confidentiality, effect on evaluations, and difficulty translating data to changes in clinical practice.

CONCLUSIONS

Residents generally desire increased dissemination of practice habit data that focuses on patient-oriented measures such as adherence to disease-specific guidelines and is both individual and team-based. Residency programs may benefit their trainees and improve resident-reported adherence to the ACGME requirement both by taking resident preferences into account and by addressing concerns about confidentiality when providing practice habit data.

Electronic Health Records as an Educational Tool: Viewpoint

BACKGROUND

Electronic health records (EHRs) have been adopted by most hospitals and medical offices in the United States. Because of the rapidity of implementation, health care providers have not been able to leverage the full potential of the EHR for enhancing clinical care, learning, and teaching. Physicians are spending an average of 49% of their working hours on EHR documentation, chart review, and other indirect tasks related to patient care, which translates into less face time with patients.

OBJECTIVE

The purpose of this article is to provide a preliminary framework to guide the use of EHRs in teaching and evaluation of residents.

METHODS

First we discuss EHR educational capabilities that have not been reviewed in sufficient detail in the literature and expand our discussion for each educational activity with examples. We emphasize quality improvement of clinical notes as a basic foundational skill using a spreadsheet-based application as an assessment tool. Next, we integrate the six Accreditation Council for Graduate Medical Education (ACGME) Core Competencies and Milestones (CCMs) framework with the Reporter-Interpreter-Manager-Educator (RIME) model to expand our assessments of other areas of resident performance related to EHR use. Finally, we discuss how clinical utility, clinical outcome, and clinical reasoning skills can be assessed in the EHR.

RESULTS

We describe a pilot conceptual framework-CCM framework-to guide and demonstrate the use of the EHR for education in a clinical setting.

CONCLUSIONS

As EHRs and other supporting technologies evolve, medical educators should continue to look for new opportunities within the EHR for education. Our framework is flexible to allow adaptation and use in most training programs. Future research should assess the validity of such methods on trainees' education.

Pediatric Case Exposure During Emergency Medicine Residency

OBJECTIVE

While emergency medicine (EM) physicians treat the majority of pediatric EM (PEM) patients in the United States, little is known about their PEM experience during training. The primary objective was to characterize the pediatric case exposure and compare to established EM residency training curricula among EM residents across five U.S. residency programs.

METHODS

We performed a multicenter medical record review of all pediatric patients (aged < 18 years) seen by the 2015 graduating resident physicians at five U.S. EM training programs. Resident-level counts of pediatric patients were measured and specific counts were classified by the 2016 Model of Clinical Practice of Emergency Medicine (MCP) and Pediatric Emergency Care Applied Research Network (PECARN) diagnostic categories. We assessed variability between residents and between programs.

RESULTS

A total of 36,845 children were managed by 68 residents across all programs. The median age was 6 years. The median number of patients per resident was 660 with an interquartile range of 336. The most common PECARN diagnostic categories were trauma, gastrointestinal, and respiratory disease. Thirty-two core MCP diagnoses (43% of MCP list) were not seen by at least 50% of the residents. We found statistically significant variability between programs in both PECARN diagnostic categories (p < 0.01) and MCP diagnoses (p < 0.01).

CONCLUSION

There is considerable variation in the number of pediatric patients and the diagnostic case volume seen by EM residents. The relationship between this case variability and competence upon graduation is unknown; further investigation is warranted to better inform program-specific curricula and to guide training requirements in EM.