Academic outcomes of a community-based longitudinal integrated clerkships program

A value analysis of longitudinal integrated clerkships: consideration of costs and benefits

Longitudinal Integrated Clerkships (LICs) prioritise longitudinal relationships with faculty, patients, and place. Research shows that LICs benefit students and faculty, but most medical schools have limited LIC programmes. This is likely due to perceptions that LICs are more costly and complex than traditional block rotations (TBRs). The perceived cost versus evidence-based value related to clerkship education has not been examined in detail. Until recently, no 'All-LIC' medical school exemplars existed in the US, limiting the value of this model as well as the ability to examine relative cost and complexity. In this paper, we draw on our experience launching three 'All-LIC' medical schools in the United States - schools in which the entire clerkship class participates in a comprehensive clerkship-year LIC. We propose that the known benefits of LICs coupled with cost-mitigation strategies related to running an 'All LIC' model for core clinical clerkships, rather than block and LIC models simultaneously, results in a higher value for medical schools.

Exploring Models of Exposure to Primary Care Careers in Training: a Narrative Review

Multiple models of clinical exposure to primary care exist within undergraduate medical education (UME) and graduate medical education (GME). In this narrative review, we explore the evidence behind these different models of exposure, their alignment with positive promoters of primary care careers, and the pros and cons of each. Without positive exposure to primary care during training, sustaining the future primary care work force becomes increasingly challenging. Here, we explore multiple models of clinical exposure in UME, including longitudinal integrated clerkships, primary care tracks, and primary care clerkships. Within GME, we will review the impact of primary care tracks, Area Health Education Centers, block scheduling models, and continuity clinic scheduling models. The goal of this narrative review is to allow educators to think broadly and intentionally about the array of models to develop positive primary care experiences and perceptions in training, ultimately sustaining the primary care workforce.

Psychological safety and accountability in longitudinal integrated clerkships: a dual institution qualitative study

BACKGROUND

Psychological safety and accountability are frameworks to describe relationships in the workplace. Psychological safety is a shared belief by members of a team that it is safe to take interpersonal risks. Accountability refers to being challenged and expected to meet expectations and goals. Psychological safety and accountability are supported by relational trust. Relational continuity is the educational construct underpinning longitudinal integrated clerkships. The workplace constructs of psychological safety and accountability may offer lenses to understand students' educational experiences in longitudinal integrated clerkships.

METHODS

We performed a qualitative study of 9 years of longitudinal integrated clerkship graduates from two regionally diverse programs-at Harvard Medical School and the University of North Carolina School of Medicine. We used deductive content analysis to characterize psychological safety and accountability from semi-structured interviews of longitudinal integrated clerkship graduates.

RESULTS

Analysis of 20 graduates' interview transcripts reached saturation. We identified 109 discrete excerpts describing psychological safety, accountability, or both. Excerpts with high psychological safety described trusting relationships and safe learning spaces. Low psychological safety included fear and frustration and perceptions of stressful learning environments. Excerpts characterizing high accountability involved increased learning and responsibility toward patients. Low accountability included students not feeling challenged. Graduates' descriptions with both high psychological safety and high accountability characterized optimized learning and performance.

CONCLUSIONS

This study used the workplace-based frameworks of psychological safety and accountability to explore qualitatively longitudinal integrated clerkship graduates' experiences as students. Graduates described high and low psychological safety and accountability. Graduates' descriptions of high psychological safety and accountability involved positive learning experiences and responsibility toward patients. The relational lenses of psychological safety and accountability may inform faculty development and future educational research in clinical medical education.

Qualitative Findings of a Nominal Group Process to Identify Critical Factors in New LIC Implementation

OBJECTIVES

Medical schools considering longitudinal integrated clerkships (LICs) have access to literature that provides recommendations for planning, implementation, and sustainability. However, LIC development and implementation remain notoriously challenging. University of Utah's LIC development process was informed by the documented experiences of long-established LIC programs. A literature gap was identified pertaining to more recently implemented LICs. The aim of this study was to explore the experiences of faculty in the early stages of LIC development.

METHODS

Thirteen representatives from eight LICs implemented after 2015 participated in 2 Zoom focus groups (5 participated in the first and the other 8 participated in the second). Participants were asked questions to assess key supports, barriers, and recommendations. Following the focus groups, participants were asked to rank the responses based on their level of importance.

RESULTS

Highest ranked supports included stakeholder and partner involvement; a dedicated coordinator or team; and strong, committed leadership. Highest ranked barriers included difficulty recruiting preceptors and clinical sites; underestimation of the amount of work required to coordinate the LIC; and challenges in providing the needed faculty development. Top recommendations for new LICs included investing in the needs of clinical partners; staffing or assigning a dedicated coordinator early in the development and implementation process; and frequent communication with all stakeholders.

CONCLUSION

Despite variation among the types of new LICs represented, there was consensus among participants on the importance of key supports, barriers, and recommendations. Knowledge of these factors can help new schools plan and allocate resources during their LIC development process. Participants found the focus group process and follow-up discussions useful and have formed an ongoing workgroup which meets quarterly.

The Perception of Surgery Program Directors of Residency Applicants From Longitudinal Integrated Curriculum Medical Schools: A Survey of the Association of Program Directors in Surgery Members

Research statement This study explores whether longitudinal integrated clerkship (LIC) students are competitive general surgery applicants and if they are perceived as adequately prepared for general surgery residency compared to traditional block rotation (BR) students. Background/relevance of the study There is increasing interest in LIC models of clinical education versus BR models. LIC students have been shown to perform similarly on examinations to BR students. However, while LICs seem well suited for students pursuing primary care specialties, little is known about how this approach impacts clinical education for surgery. Design and methods An electronic survey was prepared and approved by the Association of Program Directors in Surgery (APDS) and our university's institutional review board (IRB). Ten multiple-choice questions were administered along with an option for narrative comments. Surveys were sent over a one-month period to members of APDS Listserv. Returned emails were de-identified, and the results were tabulated. Results From 43 responses, the majority identified as program directors (PDs) (65%) and reported being somewhat familiar or very familiar with LICs (90%). When asked about the statement "LIC students are prepared for surgical residency," 22% "disagreed" or "strongly disagreed." When asked "How would you rank a LIC prospective applicant in comparison to a BR student?" 35% responded that they would rank the LIC student lower or not at all. Of the respondents, 47% reported that they have current residents who were LIC students. Most of these residents (65%) are graded as "average" for current performance. Conclusions The results suggest that medical students who are trained using LICs may be disadvantaged when applying to general surgery residencies. Interpretation is limited by the small number of respondents, and it only reflects the opinions of active APDS Listserv members. Further study is needed to confirm these findings and elucidate the basis of perceived deficiencies in LICs. Students from these schools should be advised to obtain additional surgery experience.

The Impact of Internal Medicine Clerkship Characteristics and NBME Subject Exams on USMLE Step 2 Clinical Knowledge Exam Performance

BACKGROUND

Residency program directors will likely emphasize the United States Medical Licensing Exam (USMLE) Step 2 clinical knowledge (CK) exam more during residency application given the recent USMLE Step 1 transition to pass/fail scoring. We examined how internal medicine clerkship characteristics and NBME subject exam scores affect USMLE Step 2 CK performance.

DESIGN

The authors used univariable and multivariable generalized estimating equations to determine associations between Step 2 CK performance and internal medicine clerkship characteristics and NBME subject exams. The sample had 21,280 examinees' first Step 2 CK scores for analysis.

RESULTS

On multivariable analysis, Step 1 performance (standardized β = 0.45, p < .001) and NBME medicine subject exam performance (standardized β = 0.40, p < .001) accounted for approximately 60% of the variance in Step 2 CK performance. Students who completed the internal medicine clerkship last in the academic year scored lower on Step 2 CK (M_diff = -3.17 p < .001). Students who had a criterion score for passing the NBME medicine subject exam scored higher on Step 2 CK (M_diff = 1.10, p = .03). There was no association between Step 2 CK performance and other internal medicine clerkship characteristics (all p > 0.05) nor with the total NBME subject exams completed (β=0.05, p = .78).

CONCLUSION

Despite similarities between NBME subject exams and Step 2 CK, the authors did not identify improved Step 2 CK performance for students who had more NBME subject exams. The lack of association of Step 2 CK performance with many internal medicine clerkship characteristics and more NBME subject exams has implications for future clerkship structure and summative assessment. The improved Step 2 CK performance in students that completed their internal medicine clerkship earlier warrants further study given the anticipated increase in emphasis on Step 2 CK.

Predicting United States Medical Licensing Examination Step 2 Clinical Knowledge Scores from Previous Academic Performance Measures within a Longitudinal Interleaved Curriculum

Background

United States Medical Licensing Examination (USMLE) Step 1 is a common metric looked at by residency programs to determine invitations for candidates to interview. However, USMLE Step 2 Clinical Knowledge (CK) has also been an important factor for selecting applicants to interview and plays a significant role during applicant selection. This study aims to identify academic performance measures that correlate with USMLE Step 2 CK scores and to develop a model to predict USMLE Step 2 CK scores using previous academic measures from the first two cohorts in the longitudinal interleaved clerkship (LInC) at the Kirk Kerkorian School of Medicine at the University of Nevada, Las Vegas (KSOM).

Setting

The KSOM is a newly accredited US allopathic medical school that accepted its first class in 2017. At KSOM, a LInC model is used in the primary clinical year. In this model, rotations are two weeks in duration before moving on to the next specialty. Students complete the National Board of Medical Examiners (NBME) subject examinations in all six specialties in one week at the midpoint and the end of the LInC. Students who passed an exam at the midpoint can opt out of that exam at the end as the higher of the two exam scores is recorded. However, most students choose to take all the exams again to improve their scores and prepare for USMLE Step 2 CK.

Methodology

Academic performance measures were gathered from the class of 2021 and 2022 (n = 101) including undergraduate grade point average (GPA), undergraduate science GPA, medical college admission test score, USMLE Step 1 score, NBME clinical subject exam scores, and USMLE Step 2 CK scores. Pearson correlations were run between the performance variables and USMLE Step 2 CK scores to measure influence variables individually, then a regression model measured impacts of variables together.

Results

All variables except undergraduate science GPA significantly correlated with USMLE Step 2 CK score. USMLE Step 1 had the strongest correlation (r = 0.752, p < 0.001). The regression model had an R of 0.859 with the internal medicine subject exam showing the highest beta coefficient (0.327, p < 0.001).

Conclusions

This study determined that USMLE Step 2 CK scores can be effectively predicted using available performance measures. With USMLE Step 1 becoming pass/fail in January 2022, the importance of USMLE Step 2 CK as a screening tool in the residency application process will likely increase. This study was conducted within a LInC curriculum and may have limited value in the prediction of scores within other clinical year curricula.

To the Point: Integrating the Obstetrics and Gynecology Core Clerkship into a Longitudinal Integrated Curriculum in US Medical Schools

This article, from the "To the Point" series that is prepared by the Association of Professors of Gynecology and Obstetrics (APGO) Undergraduate Medical Education Committee, is a review of issues to consider when integrating obstetrics and gynecology into a Longitudinal Integrated Clerkship (LIC) model. The background, objectives, and outcomes related to LIC are discussed. Specific challenges and solutions for structuring an LIC in the context of obstetrics and gynecology are examined, which include student/faculty issues as well as process matters such as general infrastructure and specific curriculum concerns.

Three Decades Later: A Scoping Review of the Literature Related to the United States Medical Licensing Examination

PURPOSE

To conduct a scoping review of the timing, scope, and purpose of literature related to the United States Medical Licensing Examination (USMLE) given the recent impetus to revise USMLE scoring.

METHOD

The authors searched PubMed, PsycInfo, and ERIC for relevant articles published from 1990 to 2019. Articles selected for review were labeled as research or commentaries and coded by USMLE Step level, sample characteristics (e.g., year(s), single/multiple institutions), how scores were used (e.g., predictor/outcome/descriptor), and purpose (e.g., clarification/justification/description).

RESULTS

Of the 741 articles meeting inclusion criteria, 636 were research and 105 were commentaries. Publication totals in the past 5 years exceeded those of the first 20 years.Step 1 was the sole focus of 38%, and included in 84%, of all publications. Approximately half of all research articles used scores as a predictor or outcome measure related to other curricular/assessment efforts, with a marked increase in the use of scores as predictors in the past 10 years. The overwhelming majority of studies were classified as descriptive in purpose.

CONCLUSIONS

Nearly 30 years after the inception of the USMLE, aspirations for its predictive utility are rising faster than evidence supporting the manner in which the scores are used. A closer look is warranted to systematically review and analyze the contexts and purposes for which USMLE scores can productively be used. Future research should explore cognitive and noncognitive factors that can be used in conjunction with constrained use of USMLE results to inform evaluation of medical students and schools and to support the residency selection process.

Which Internal Medicine Clerkship Characteristics Are Associated With Students' Performance on the NBME Medicine Subject Exam? A Multi-Institutional Analysis

PURPOSE

To identify which internal medicine clerkship characteristics may relate to NBME Medicine Subject Examination scores, given the growing trend toward earlier clerkship start dates.

METHOD

The authors used linear mixed effects models (univariable and multivariable) to determine associations between medicine exam performance and clerkship characteristics (longitudinal status, clerkship length, academic start month, ambulatory clinical experience, presence of a study day, involvement in a combined clerkship, preclinical curriculum type, medicine exam timing). Additional covariates included number of NBME clinical subject exams used, number of didactic hours, use of a criterion score for passing the medicine exam, whether medicine exam performance was used to designate clerkship honors, and United States Medical Licensing Examination Step 1 performance. The sample included 24,542 examinees from 62 medical schools spanning 3 academic years (2011-2014).

RESULTS

The multivariable analysis found no significant association between clerkship length and medicine exam performance (all pairwise P > .05). However, a small number of examinees beginning their academic term in January scored marginally lower than those starting in July (P < .001). Conversely, examinees scored higher on the medicine exam later in the academic year (all pairwise P < .001). Examinees from schools that used a criterion score for passing the medicine exam also scored higher than those at schools that did not (P < .05). Step 1 performance remained positively associated with medicine exam performance even after controlling for all other variables in the model (P < .001).

CONCLUSIONS

In this sample, the authors found no association between many clerkship variables and medicine exam performance. Instead, Step 1 performance was the most powerful predictor of medicine exam performance. These findings suggest that medicine exam performance reflects the overall medical knowledge students accrue during their education rather than any specific internal medicine clerkship characteristics.

Self-Perceptions of Readiness to Use Electronic Health Records Among Medical Students: Survey Study

BACKGROUND

Although several national organizations have declared the ability to work with electronic health records (EHRs) as a core competency of medical education, EHR education and use among medical students vary widely. Previous studies have reported EHR tasks performed by medical students, but students' self-perceived readiness and comfort with EHRs are relatively unknown.

OBJECTIVE

This study aimed to better understand medical students' self-perceived readiness to use EHRs to identify potential curricular gaps and inform future training efforts based on students' perspectives.

METHODS

The authors deployed a survey investigating self-perceived comfort with EHRs at 2 institutions in the United States in May 2019. Descriptive statistics were generated regarding demographics, comfort level with various EHR-related tasks, and cross-institutional comparisons. We also assessed the impact of extracurricular EHR experience on comfort level.

RESULTS

In total, 147 medical students responded, of which 80 (54.4%) were female, with equal distribution across all 4 years of training. Overall confidence was generally higher for students with longer extracurricular EHR experience, even when adjusted for age, gender, year of training, and institution. Students were most comfortable with tasks related to looking up information in the EHR and felt less comfortable with tasks related to entering new information and managing medications. Fourth-year students at both schools reported similar levels of comfort with EHR use, despite differences in preclinical EHR training. Open-ended comments emphasized the value of experiential training over didactic formats.

CONCLUSIONS

Information entry and medication management in the EHR represent areas for future curricular development. Experiential training via extracurricular activities and early clinical exposure may be high-yield approaches to help medical students achieve critical EHR competencies.

Career destinations of graduates from a medical school with an 18-week longitudinal integrated clerkship in general practice: a survey of alumni 6 to 8 years after graduation

Background

There is a worldwide recruitment and retention crisis in general practice. Workforce planning has identified the need to train more general practitioners as an urgent priority. Exposure of medical students to general practice as part of the formal and hidden curriculum, the use of longitudinal integrated clerkships, and positive experiences and role models in general practice are all thought to be contributing factors to doctors choosing careers in general practice.

Aim

The aim of this study was to identify career destinations of medical school graduates in a medical school with an 18-week longitudinal integrated clerkship in general practice.

Design and setting

This study was conducted in a single graduate entry medical school at the University of Limerick, Ireland.

Participants

Medical school alumni 6–8 years after graduation.

Method

A survey of graduating cohorts of the medical school from 2011 to 2013 was conducted through email and telephone.

Results

There were a total of 175 alumni for the period 2011 to 2013. Data was collected on 92% (161/175) through an online survey, follow-up email and telephone interview, and was triangulated with searches of professional registration databases and information from key informants. Between 6 and 8 years after graduation, a total of 43% of alumni were engaged in general practice as a career.

Conclusion

The reform of the delivery of general practice within medical school curricula should be considered by medical schools, curriculum designers and policy-makers as part of an overall strategy to address the recruitment and retention of general practitioners as part of the global healthcare workforce.

A Narrative Literature Review Considering the Development and Implementation of Longitudinal Integrated Clerkships, Including a Practical Guide for Application

Hailed by supporters as the answer to many challenges facing medical schools and the wider health care system, longitudinal integrated clerkships (LICs) offer a practical and sustainable alternative to more traditional block rotational models. Given this, their popularity as a curricular measure is increasing, although such clerkships remain relatively novel within the United Kingdom. This narrative literature review of international work provides a comprehensive introduction to developing and implementing LICs within medical education. This review generates a practical guide for medical educators with a focus on the development and implementation of LICs within the United Kingdom, on which there is little work. Using illustrated examples and with reference to contemporary literature, it outlines the rationale for considering an LIC within a curriculum, the different types of LIC, barriers and enabling factors to LIC implementation and considers the contemporary application of LIC models within the United Kingdom. The practical guide details key questions educators must consider when developing and implementing an LIC, particularly within the landscape of UK medical education.

Graduates' Perceptions of Learning Affordances in Longitudinal Integrated Clerkships: A Dual-Institution, Mixed-Methods Study

PURPOSE

The authors explored affordances that contribute to participants' successful learning in longitudinal integrated clerkships (LICs).

METHOD

This dual-institutional, mixed-methods study included electronic surveys and semistructured interviews of LIC graduates who completed their core clinical (third) year of medical school. These LIC graduates took part in LICs at Harvard Medical School from 2004 to 2013 and the University of North Carolina School of Medicine-Asheville campus from 2009 to 2013. The survey questions asked LIC graduates to rate components of LICs that they perceived as contributing to successful learning. A research assistant interviewed a subset of study participants about their learning experiences. The authors analyzed aggregate data quantitatively and performed a qualitative content analysis on interview data.

RESULTS

The graduates reported multiple affordances that they perceive contributed to successful learning in their LIC. The most reported components included continuity and relationships with preceptors, patients, place, and peers, along with integration of and flexibility within the curriculum.

CONCLUSIONS

As LIC models grow in size and number, and their structures and processes evolve, learners' perceptions of affordances may guide curriculum planning. Further research is needed to investigate to what degree and by what means these affordances support learning in LICs and other models of clinical education.

Robust (and Ethical) Educational Research Designs

Educational research can be quite confusing to individuals without formal training in educational research methods. Many educators trained in "high-consensus" fields such as the health and medical sciences often expect to utilize the same esteemed experimental and quasi-experimental designs in their educational research efforts, but are surprised to learn that this is often impossible for ethical reasons. This article presents five robust research designs that generally are compatible with education studies, and discusses how each design may be used in a variety of educational research scenarios.

Effectiveness of a rural longitudinal integrated clerkship in preparing medical students for internship

BACKGROUND

We interviewed graduates from the first two cohorts of a postgraduate medical program that had a senior year longitudinal integrated clerkship (LIC) in a practice setting in rural New South Wales, Australia to determine how well their training prepared them to be junior doctors (3-4 years after graduation), and what aspects of that training they thought were particularly useful.

METHODS

In-depth interviews.

RESULTS

Fourteen junior doctors were interviewed. Participants reported feeling well prepared in ability to develop close relationships with clinical supervisors, good clinical and procedural skills, ability to work autonomously and work in teams, knowledge of health systems, ability to ensure self-care, and professionalism. Consensus view was that a rural placement was an excellent way to learn medicine for a variety of reasons including relationships with clinicians, less competition for access to patients, and opportunities to extend their clinical skills and act up to intern level.

CONCLUSION

The advantages we found in the training these junior doctors received which prepared them well for internship were integral both to the longitudinal, unstructured placement, and to the fact that it was carried out in a rural area. The two aspects of these placements appear to act synergistically, reinforcing the learning experience.