Intersite consistency as a measurement of programmatic evaluation in a medicine clerkship with multiple, geographically separated sites

Evaluating Intersite Consistency Across 11 Geographically Distinct Pediatric Clerkship Training Sites: Providing Assurance That Educational Comparability Is Possible

INTRODUCTION

We compared core pediatric clerkship student assessments across 11 geographically distinct learning environments following a major curriculum change. We sought to determine if intersite consistency existed, which can be used as a marker of program evaluation success.

METHODS

We evaluated students' overall pediatric clerkship performance along with individual assessments that target our clerkship learning objectives. Using the data of graduating classes from 2015 to 2019 (N = 859), we conducted an analysis of covariance and multivariate logistic regression analysis to investigate whether the performance varied across training sites.

RESULTS

Of the students, 833 (97%) were included in the study. The majority of the training sites did not show statistically significant differences from each other. After controlling for the Medical College Admission Test total score and the average pre-clerkship National Board of Medical Examiners final exam score, the clerkship site only explained a 3% additional variance of the clerkship final grade.

CONCLUSIONS

Over the ensuing 5-year period after a curriculum overhaul to an 18-month, integrated module pre-clerkship curriculum, we found that student pediatric clerkship performance in clinical knowledge and skills did not differ significantly across 11 varied geographic teaching sites when controlling for students' pre-clerkship achievement. Specialty-specific curriculum resources, faculty development tools, and assessment of learning objectives may provide a framework for maintaining intersite consistency when faced with an expanding network of teaching facilities and faculty.

Evaluating the Reliability and Validity Evidence of the RIME (Reporter-Interpreter-Manager-Educator) Framework for Summative Assessments Across Clerkships

PURPOSE

The ability of medical schools to accurately and reliably assess medical student clinical performance is paramount. The RIME (reporter-interpreter-manager-educator) schema was originally developed as a synthetic and intuitive assessment framework for internal medicine clerkships. Validity evidence of this framework has not been rigorously evaluated outside of internal medicine. This study examined factors contributing to variability in RIME assessment scores using generalizability theory and decision studies across multiple clerkships, thereby contributing to its internal structure validity evidence.

METHOD

Data were collected from RIME-based summative clerkship assessments during 2018-2019 at Virginia Commonwealth University. Generalizability theory was used to explore variance attributed to different facets through a series of unbalanced random-effects models by clerkship. For all analyses, decision (D-) studies were conducted to estimate the effects of increasing the number of assessments.

RESULTS

From 231 students, 6,915 observations were analyzed. Interpreter was the most common RIME designation (44.5%-46.8%) across all clerkships. Variability attributable to students ranged from 16.7% in neurology to 25.4% in surgery. D-studies showed the number of assessments needed to achieve an acceptable reliability (0.7) ranged from 7 in pediatrics and surgery to 11 in internal medicine and 12 in neurology. However, depending on the clerkship each student received between 3 and 8 assessments.

CONCLUSIONS

This study conducted generalizability- and D-studies to examine the internal structure validity evidence of RIME clinical performance assessments across clinical clerkships. Substantial proportion of variance in RIME assessment scores was attributable to the rater, with less attributed to the student. However, the proportion of variance attributed to the student was greater than what has been demonstrated in other generalizability studies of summative clinical assessments. Overall, these findings support the use of RIME as a framework for assessment across clerkships and demonstrate the number of assessments required to obtain sufficient reliability.

Exploring the Predictors of Post-Clerkship USMLE Step 1 Scores

Theory: We used two theoretical frameworks for this study: a) experiential learning, whereby learners construct new knowledge based on prior experience, and learning grows out of a continuous process of reconstructing experience, and b) deliberate practice, whereby the use of testing (test-enhanced learning) promotes learning and produces better long-term retention. Hypothesis: We hypothesized that moving the USMLE Step 1 exam to follow the clerkship year would provide students with a context for basic science learning that may enhance exam performance. We also hypothesized that examination performance variables, specifically National Board of Medical Examiners (NBME) Customized Basic Science Examinations and NBME subject examinations in clinical disciplines would account for a moderate to large amount of the variance in Step 1 scores. Thus we examined predictors of USMLE Step 1 scores when taken after the core clerkship year. Method: In 2011, we revised our medical school curriculum and moved the timing of Step 1 to follow the clerkship year. We performed descriptive statistics, an ANCOVA to compare Step 1 mean scores for three graduating classes of medical students before and after the curriculum changes, and stepwise linear regression to investigate the association between independent variables and the primary outcome measure after curriculum changes. Results: 993 students took the Step 1 exam, which included graduating classes before (2012-2014, N = 491) and after (2015-2017, N = 502) the curriculum change. Step 1 scores increased significantly following curricular revision (mean 218, SD 18.2, vs. 228, SD 16.7, p < 0.01) after controlling for MCAT and undergraduate GPA. Overall, 66.4% of the variance in Step 1 scores after the clerkship year was explained by: the mean score on fourteen pre-clerkship customized NBME exams (p < 0.01, 57.0% R²); performance on the surgery NBME subject exam (p < 0.01, 3.0% R²); the pediatrics NBME subject exam (p < 0.01, 2.0% R²); the Comprehensive Basic Science Self-Assessment (p < .01, 2.0% R²) ; the internal medicine NBME subject exam (p < 0.01, 0.03% R²), pre-clerkship Integrated Clinical Skills score (p < 0.01, 0.05% R²), and the pre-matriculation MCAT (p < 0.01, 0.01% R²). Conclusion: In our institution, nearly two-thirds of the variance in performance on Step 1 taken after the clerkship year was explained mainly by pre-clerkship variables, with a smaller contribution emanating from clerkship measures. Further study is needed to uncover the specific aspects of the clerkship experience that might contribute to success on high stakes licensing exam performance.

Exploring Perspectives from Internal Medicine Clerkship Directors in the USA on Effective Narrative Evaluation: Results from the CDIM National Survey

PURPOSE

Clinical performance evaluations play a critical role in determining medical school clerkship grades. This study aimed to provide clarification from clerkship directors in internal medicine on what constitutes an effective and informative narrative description of student performance.

METHODS

In September 2016, the Clerkship Directors in Internal Medicine (CDIM) electronically administered its annual, voluntary, and confidential cross-sectional survey of its US membership. One section of the survey asked six questions regarding the helpful components of an effective narrative evaluation. Respondents were asked to rate the effectiveness of elements contained within narrative evaluations of students.

RESULTS

Ninety-five CDIM members responded to the survey with an overall response rate of 74.2%. Descriptions of skills and behaviors were felt to be the most important, followed by a description of the overall synthetic or global assessment level of the student. Descriptions of personality and attitude were the next highest rated feature followed by adjectives describing performance. Length was felt to be the least important component. In free-text comments, several respondents indicated that direct observation of performance and specific examples of skills and behaviors are also desirable.

CONCLUSIONS

Narrative evaluations of students that explicitly comment on skills, behaviors, and an overarching performance level of the learner are strongly preferred by clerkship directors. Direct observation of clinical performance and giving specific examples of such behaviors give evaluations even more importance. Faculty development on evaluation and assessment should include instruction on these narrative assessment characteristics.

Validity Evidence and Scoring Guidelines for Standardized Patient Encounters and Patient Notes From a Multisite Study of Clinical Performance Examinations in Seven Medical Schools

PURPOSE

To examine validity evidence of local graduation competency examination scores from seven medical schools using shared cases and to provide rater training protocols and guidelines for scoring patient notes (PNs).

METHOD

Between May and August 2016, clinical cases were developed, shared, and administered across seven medical schools (990 students participated). Raters were calibrated using training protocols, and guidelines were developed collaboratively across sites to standardize scoring. Data included scores from standardized patient encounters for history taking, physical examination, and PNs. Descriptive statistics were used to examine scores from the different assessment components. Generalizability studies (G-studies) using variance components were conducted to estimate reliability for composite scores.

RESULTS

Validity evidence was collected for response process (rater perception), internal structure (variance components, reliability), relations to other variables (interassessment correlations), and consequences (composite score). Student performance varied by case and task. In the PNs, justification of differential diagnosis was the most discriminating task. G-studies showed that schools accounted for less than 1% of total variance; however, for the PNs, there were differences in scores for varying cases and tasks across schools, indicating a school effect. Composite score reliability was maximized when the PN was weighted between 30% and 40%. Raters preferred using case-specific scoring guidelines with clear point-scoring systems.

CONCLUSIONS

This multisite study presents validity evidence for PN scores based on scoring rubric and case-specific scoring guidelines that offer rigor and feedback for learners. Variability in PN scores across participating sites may signal different approaches to teaching clinical reasoning among medical schools.

Regular Formal Evaluation Sessions are Effective as Frame-of-Reference Training for Faculty Evaluators of Clerkship Medical Students

BACKGROUND

Face-to-face formal evaluation sessions between clerkship directors and faculty can facilitate the collection of trainee performance data and provide frame-of-reference training for faculty.

OBJECTIVE

We hypothesized that ambulatory faculty who attended evaluation sessions at least once in an academic year (attendees) would use the Reporter-Interpreter-Manager/Educator (RIME) terminology more appropriately than faculty who did not attend evaluation sessions (non-attendees).

DESIGN

Investigators conducted a retrospective cohort study using the narrative assessments of ambulatory internal medicine clerkship students during the 2008-2009 academic year.

PARTICIPANTS

The study included assessments of 49 clerkship medical students, which comprised 293 individual teacher narratives.

MAIN MEASURES

Single-teacher written and transcribed verbal comments about student performance were masked and reviewed by a panel of experts who, by consensus, (1) determined whether RIME was used, (2) counted the number of RIME utterances, and (3) assigned a grade based on the comments. Analysis included descriptive statistics and Pearson correlation coefficients.

KEY RESULTS

The authors reviewed 293 individual teacher narratives regarding the performance of 49 students. Attendees explicitly used RIME more frequently than non-attendees (69.8 vs. 40.4 %; p < 0.0001). Grades recommended by attendees correlated more strongly with grades assigned by experts than grades recommended by non-attendees (r = 0.72; 95 % CI (0.65, 0.78) vs. 0.47; 95 % CI (0.26, 0.64); p = 0.005). Grade recommendations from individual attendees and non-attendees each correlated significantly with overall student clerkship clinical performance [r = 0.63; 95 % CI (0.54, 0.71) vs. 0.52 (0.36, 0.66), respectively], although the difference between the groups was not statistically significant (p = 0.21).

CONCLUSIONS

On an ambulatory clerkship, teachers who attended evaluation sessions used RIME terminology more frequently and provided more accurate grade recommendations than teachers who did not attend. Formal evaluation sessions may provide frame-of-reference training for the RIME framework, a method that improves the validity and reliability of workplace assessment.

Are Commonly Used Premedical School or Medical School Measures Associated With Board Certification?

Purpose: To determine if there is an association between several commonly obtained premedical school and medical school measures and board certification performance. We specifically included measures from our institution for which we have predictive validity evidence into the internship year. We hypothesized that board certification would be most likely to be associated with clinical measures of performance during medical school, and with scores on standardized tests, whether before or during medical school. Methods: Achieving board certification in an American Board of Medical Specialties specialty was used as our outcome measure for a 7-year cohort of graduates (1995–2002). Age at matriculation, Medical College Admissions Test (MCAT) score, undergraduate college grade point average (GPA), undergraduate college science GPA, Uniformed Services University (USU) cumulative GPA, USU preclerkship GPA, USU clerkship year GPA, departmental competency committee evaluation, Internal Medicine (IM) clerkship clinical performance rating (points), IM total clerkship points, history of Student Promotion Committee review, and United States Medical Licensing Examination (USMLE) Step 1 score and USMLE Step 2 clinical knowledge score were associated with this outcome. Results: Ninety-three of 1,155 graduates were not certified, resulting in an average rate of board certification of 91.9% for the study cohort. Significant small correlations were found between board certification and IM clerkship points (r = 0.117), IM clerkship grade (r = 0.108), clerkship year GPA (r = 0.078), undergraduate college science GPA (r = 0.072), preclerkship GPA and medical school GPA (r = 0.068 for both), USMLE Step 1 (r = 0.066), undergraduate college total GPA (r = 0.062), and age at matriculation (r = −0.061). In comparing the two groups (board certified and not board certified cohorts), significant differences were seen for all included variables with the exception of MCAT and USMLE Step 2 clinical knowledge scores. All the variables put together could explain 4.1% of the variance of board certification by logistic regression. Conclusions: This investigation provides some additional validity evidence that measures collected for purposes of student evaluation before and during medical school are warranted.

The Association of Students Requiring Remediation in the Internal Medicine Clerkship With Poor Performance During Internship

Purpose: To determine whether the Uniformed Services University (USU) system of workplace performance assessment for students in the internal medicine clerkship at the USU continues to be a sensitive predictor of subsequent poor performance during internship, when compared with assessments in other USU third year clerkships. Method: Utilizing Program Director survey results from 2007 through 2011 and U.S. Medical Licensing Examination (USMLE) Step 3 examination results as the outcomes of interest, we compared performance during internship for students who had less than passing performance in the internal medicine clerkship and required remediation, against students whose performance in the internal medicine clerkship was successful. We further analyzed internship ratings for students who received less than passing grades during the same time period on other third year clerkships such as general surgery, pediatrics, obstetrics and gynecology, family medicine, and psychiatry to evaluate whether poor performance on other individual clerkships were associated with future poor performance at the internship level. Results for this recent cohort of graduates were compared with previously published findings. Results: The overall survey response rate for this 5 year cohort was 81% (689/853). Students who received a less than passing grade in the internal medicine clerkship and required further remediation were 4.5 times more likely to be given poor ratings in the domain of medical expertise and 18.7 times more likely to demonstrate poor professionalism during internship. Further, students requiring internal medicine remediation were 8.5 times more likely to fail USMLE Step 3. No other individual clerkship showed any statistically significant associations with performance at the intern level. On the other hand, 40% of students who successfully remediated and did graduate were not identified during internship as having poor performance. Conclusions: Unsuccessful clinical performance which requires remediation in the third year internal medicine clerkship at Uniformed Services University of the Health Sciences continues to be strongly associated with poor performance at the internship level. No significant associations existed between any of the other clerkships and poor performance during internship and Step 3 failure. The strength of this association with the internal medicine clerkship is most likely because of an increased level of sensitivity in detecting poor performance.

Novel Examination for Evaluating Medical Student Clinical Reasoning: Reliability and Association With Patients Seen

Background: Medical students learn clinical reasoning, in part, through patient care. Although the numbers of patients seen is associated with knowledge examination scores, studies have not demonstrated an association between patient problems and an assessment of clinical reasoning. Aim: To examine the reliability of a clinical reasoning examination and investigate whether there was association between internal medicine core clerkship students' performance on this examination and the number of patients they saw with matching problems during their internal medicine clerkship. Methods: Students on the core internal medicine clerkship at the Uniformed Services University students log 11 core patient problems based on the Clerkship Directors in Internal Medicine curriculum. On a final clerkship examination (Multistep), students watch a scripted video encounter between physician and patient actors that assesses three sequential steps in clinical reasoning: Step One focuses on history and physical examination; Step Two, students write a problem list after viewing additional clinical findings; Step Three, students complete a prioritized differential diagnosis and treatment plan. Each Multistep examination has three different cases. For graduating classes 2010–2012 (n = 497), we matched the number of patients seen with the problem most represented by the Multistep cases (epigastric pain, generalized edema, monoarticular arthritis, angina, syncope, pleuritic chest pain). We report two-way Pearson correlations between the number of patients students reported with similar problems and the student's percent score on: Step One, Step Two, Step Three, and Overall Test. Results: Multistep reliability: Step 1, 0.6 to 0.8; Step 2, 0.41 to 0.65; Step 3, 0.53 to 0.78; Overall examination (3 cases): 0.74 to 0.83. For three problems, the number of patients seen had small to modest correlations with the Multistep Examination of Analytic Ability total score (r = 0.27 for pleuritic pain, p &lt; 0.05, n = 81 patients; r = 0.14 for epigastric pain, p &lt; 0.05, n = 324 patients; r = 0.19 for generalized edema, p &lt; 0.05, n = 118 patients). Discussion or Conclusion: Although a reliable assessment, student performance on a clinical reasoning examination was weakly associated with the numbers of patients seen with similar problems. This may be as a result of transfer of knowledge between clinical and examination settings, the complexity of clinical reasoning, or the limits of reliability with patient logs and the Multistep.

Frameworks for learner assessment in medicine: AMEE Guide No. 78

In any evaluation system of medical trainees there is an underlying set of assumptions about what is to be evaluated (i.e., which goals reflect the values of the system or institution), what kind of observations or assessments are useful to allow judgments 1 ; and how these are to be analyzed and compared to a standard of what is to be achieved by the learner. These assumptions can be conventionalized into a framework for evaluation. Frameworks encompass, or "frame," a group of ideas or categories to reflect the educational goals against which a trainee's level of competence or progress is gauged. Different frameworks provide different ways of looking at the practice of medicine and have different purposes. In the first place, frameworks should enable educators to determine to what extent trainees are ready for advancement, that is, whether the desired competence has been attained. They should provide both a valid mental model of competence and also terms to describe successful performance, either at the end of training or as milestones during the curriculum. Consequently, such frameworks drive learning by providing learners with a guide for what is expected. Frameworks should also enhance consistency and reliability of ratings across staff and settings. Finally, they determine the content of, and resources needed for, rater training to achieve consistency of use. This is especially important in clinical rotations, in which reliable assessments have been most difficult to achieve. Because the limitations of workplace-based assessment have persisted despite the use of traditional frameworks (such as those based on knowledge, skills, and attitudes), this Guide will explore the assumptions and characteristics of traditional and newer frameworks. In this AMEE Guide, we make a distinction between analytic, synthetic, and developmental frameworks. Analytic frameworks deconstruct competence into individual pieces, to evaluate each separately. Synthetic frameworks attempt to view competence holistically, focusing evaluation on the performance in real-world activities. Developmental frameworks focus on stages of, or milestones, in the progression toward competence. Most frameworks have one predominant perspective; some have a hybrid nature.

Does the authenticity of preclinical teaching format affect subsequent clinical clerkship outcomes? A prospective randomized crossover trial

BACKGROUND

Educational theories predict conflicting results for the effect of increasing the authenticity of the teaching format of complex information on educational outcomes. We sought to determine the effect of increasingly authentic small-group, preclerkship teaching format on clerkship outcomes to further enlighten this debate.

SUMMARY

Students enrolled in a prospective randomized crossover trial that involved three content areas. For each content area, three teaching formats were tested. Participants were randomized to teaching format by content area. Clerkship outcomes were performance on an objective structured clinical exam, a DVD exam, internal medicine clerkship grades, and performance on the subject examination. The data were analyzed using a multivariate analysis of covariance. One hundred and thirty-three (78%) students participated. Teaching format did not have a statistically significant effect on any of the specified clerkship outcomes. However, number of patients seen was significantly associated with higher scores in respective outcomes by topic.

CONCLUSIONS

Second-year teaching format did not directly influence subsequent clerkship performance. Our study adds to the literature by demonstrating that the authenticity of preclinical teaching format does not appear to matter for clerkship performance; however, the number of actual patients seen does appear to influence related clerkship outcomes.

Use of a geriatric home visit experience to teach medical students the functional status assessment

INTRODUCTION

Functional status measures strongly predict hospital outcomes and mortality, yet teaching of these measures is often missing from medical schools' curricula. To address this deficiency, we developed a Geriatric Home-based Assessment (GHA) module for third-year medical students. The module was composed of a workshop and two to three home visits.

OBJECTIVE

To determine whether the GHA module would improve students' knowledge and proficiency in the functional status assessment.

PROGRAM EVALUATION

Students completed a validated questionnaire and evaluated a standardized patient in an Observed Structured Clinical Examination (OSCE). Scores from students completing the GHA were compared to the scores of students without this experience.

RESULTS

Thirty-one students participated in the GHA module, and 19 students were in the control group. The mean score on the written assessment was 87% among GHA students vs. 46% in the control group (p < 0.001). The mean clinical examination score of the intervention group was also better than that of the control group (76% vs. 46%, p < 0.001).

CONCLUSIONS

Our GHA module was effective in improving students' knowledge and proficiency in the functional status assessment. "Hands on" experiences like the GHA allow students to develop a solid foundation for assessing functional status and mobility.

Pilot study of a 'RIME'-based tool for giving feedback in a multi-specialty longitudinal clerkship

CONTEXT

The Reporter-Interpreter-Manager-Educator (RIME) evaluation framework is intuitive and reliable. Our preceptors' frustration with using summative tools for formative feedback and the hypothesis that the RIME vocabulary might improve students' and preceptors' experiences with feedback prompted us to develop and pilot a RIME-based feedback tool.

METHODS

The tool was based on the RIME vocabulary, which has previously been used for evaluation. As interpersonal skills and professionalism are difficult areas in which to give feedback, we added these as explicit categories. We piloted the tool in a longitudinal, 5-month, multi-specialty clerkship. Preceptors completed pre- and post-introductory workshop surveys. Students completed post-workshop and post-clerkship surveys.

RESULTS

Preceptors (n = 14) and students (n = 8) preferred RIME-based feedback to 'usual feedback' (previously given using end-of-clerkship evaluation forms). After the initial workshop, preceptors expected that giving feedback, including critical feedback, would be easier. After the 5-month clerkship, students reported receiving more feedback than in previous clerkships and rated feedback given using this tool more highly (P = 0.002; effect size 1.2). Students also felt it helped them understand specifically how to improve their performance (P = 0.003; effect size 1.2).

DISCUSSION

In this pilot study, preceptors and students preferred feedback with a specific RIME-based tool. Students felt such feedback was more useful and helped them identify specifically how to improve. Whether this method can improve student performance through improved feedback remains an area for further research.

The association of student examination performance with faculty and resident ratings using a modified RIME process

BACKGROUND

RIME is a descriptive framework in which students and their teachers can gauge progress throughout a clerkship from R (reporter) to I (interpreter) to M (manager) to E (educator). RIME, as described in the literature, is complemented by residents and attending physicians meeting with a clerkship director to discuss individual student progress, with group discussion resulting in assignment of a RIME stage.

OBJECTIVE

1) to determine whether a student's RIME rating is associated with end-of-clerkship examination performance; and 2) to determine whose independent RIME rating is most predictive of a student's examination performance: attendings, residents, or interns.

DESIGN

Prospective cohort study.

PARTICIPANTS

Third year medical students from academic years 2004-2005 and early 2005-2006 at 1 medical school.

MEASUREMENTS AND MAIN RESULTS

Each attending, resident, and intern independently assessed the student's final RIME stage attained. For the purpose of analysis, R stage=1, I=2, M=3, and E=4. Regression analyses were performed with examination scores as dependent variables (National Board of Medical Examiners [NBME] medicine subject examination and a clinical performance examination [CPE]), with independent variables of mean attending RIME score, mean resident score, and mean intern score. For the 122 students, significant predictors of NBME subject exam score were resident RIME rating (p = .008) and intern RIME rating (p = .02). Significant predictor of CPE performance was resident RIME rating (p = .01).

CONCLUSION

House staff RIME ratings of students are associated with student performance on written and clinical skills examinations.

Evaluation, grading, and use of the RIME vocabulary on internal medicine clerkships: results of a national survey and comparison to other clinical clerkships

BACKGROUND

Evaluation methods within and across clerkships are rapidly evolving, including greater emphasis or frameworks for descriptive evaluation and direct observation of competence.

PURPOSE

The purpose of this study is to describe current evaluation methods, use of the Reporter-Interpreter-Manager/Educator (RIME) framework, and grade assignment by internal medicine clerkship directors.

METHODS

In 2005, the Clerkship Directors in Internal Medicine surveyed its 109 institutional members. Topics included evaluation methods and grade contribution, use of evaluation sessions and/or RIME, and grade assignment (criterion referenced or normative).

RESULTS

Response rate was 81% (88/109). The evaluation methods were as follows: teachers' evaluations, 93% (64% of grade); National Board of Medical Examiners subject examination, 81% (25% of grade); faculty written exam, 34% (14% of grade); objective structured clinical examinations, 32% (12% of grade); direct observation, 22% (7% of grade). RIME is used by 42% of respondents. Many clerkship directors (43%) meet with teachers to discuss student performance. Criterion-referenced grading is used by 59%, and normative grading is used by 27%. Unsatisfactory grades are given for examination failures (72%), unprofessional behavior (49%), poor clinical performance (42%), and failure to meet requirements (18%).

CONCLUSIONS

Internal medicine clerkship directors emphasize description and observation of students. RIME and discussions with teachers are becoming commonplace.

The structure of program evaluation: an approach for evaluating a course, clerkship, or components of a residency or fellowship training program

BACKGROUND

Directors of courses, clerkships, residencies, and fellowships are responsible not only for determining whether individual trainees have met educational goals but also for ensuring the quality of the training program itself. The purpose of this article is to discuss a framework for program evaluation that has sufficient rigor to satisfy accreditation requirements yet is flexible and responsive to the uniqueness of individual educational programs.

SUMMARY

We discuss key aspects of program evaluation to include cardinal definitions, measurements, needed resources, and analyses of qualitative and quantitative data. We propose a three-phase framework for data collection (Before, During, and After) that can be used across undergraduate, graduate, and continuing medical education.

CONCLUSIONS

This Before, During, and After model is a feasible and practical approach that is sufficiently rigorous to allow for conclusions that can lead to action. It can be readily implemented for new and existing medical education programs.

A shared professional framework for anatomy and clinical clerkships

Threats to the professionalism of medical practice in the United States have resulted in an intense focus by educational organizations on what professionalism is, on how to define it, and how to evaluate it. This essay discusses alternative educational frameworks in which professionalism can be located. As the traditional analytic framework (knowledge, skills, and attitudes) and developmental frameworks are more familiar, emphasis will be placed on a "synthetic" framework that expresses a student's progress as "reporter," "interpreter," and "manager/educator." This "RIME" framework attempts to capture the classic rhythm of observation-reflection-action that is familiar to all scientists and clinicians, and attempts to express in less generic, more behavioral terms how skills, knowledge, and attitudes must all be brought to bear at the same time by a successful student. It is argued that the complexity of professional development can be embraced with simplicity, without being simplistic.

Clerkship sharing on inpatient internal medicine rotations: an emerging clerkship model

BACKGROUND

"Clerkship sharing" is the coexistence of students from 2 or more medical schools rotating on the same specialty at a single clerkship site.

PURPOSE

Clerkship sharing was characterized by answering three related questions regarding the prevalence of clerkship sharing on internal medicine inpatient rotations, stakeholders views of the advantages and disadvantages of clerkship sharing, and the ways that clerkship sharing affects medical student outcomes at an institution.

METHODS

In 2001, the Clerkship Directors in Internal Medicine (CDIM) surveyed its members; 1 section addressed clerkship sharing on inpatient rotations. In addition, the authors surveyed a convenience sample of teachers and learners at 41% of schools with clerkship sharing. Finally, using a 10-year database from one institution, we searched for differences in clerkship outcomes among students who rotated at clerkship sites with or without clerkship sharing.

RESULTS

The overall clerkship director (CD) survey response rate was 78% (96/123); 22 of 96 (23%) of CDs reported having clerkship sharing on inpatient rotations. Advantages reported included a greater diversity of clinical exposure for students (77%) and a fostering of collegial relationships (73%). We also collected 79 teacher and 77 medical student surveys from 9 (41%) medical schools identified as having clerkship sharing. The majority of these teachers and learners believed that sharing improves teaching and the overall rotation quality. All surveyed groups were concerned that clerkship sharing affected the clarity of clerkship goals, objectives, and grading. However, clerkship outcomes from 1 institution demonstrated no effect of clerkship sharing on faculty ratings of students or student examination performance.

CONCLUSIONS

Clerkship sharing appears to be an emerging clerkship model, and, although it may have inherent advantages that benefit student education, CDs should address challenges such as common goals and expectations for students and teachers.