Assessment of a surgery clerkship's performance evaluation system

Perceptions of evaluation in longitudinal versus traditional clerkships

OBJECTIVES

Methods for evaluating student performance in clerkships traditionally suffer shortcomings, partly as a result of clerkship structure. The purpose of this study was to compare preceptors' and students' perceptions of student evaluation in block clerkships and longitudinal integrated clerkships (LICs).

METHODS

From 2007 to 2009, preceptors who taught on both block clerkships and an LIC were surveyed on their perceptions of clerkship evaluation. Year 3 students were surveyed on their perceptions of clerkship evaluation at the year end. Responses from preceptors who completed both block clerkship and LIC surveys were compared using paired-samples t-test; student responses were compared using independent-samples t-test.

RESULTS

Overall, 66% (67/102) of block clerkship and 75% (77/102) of LIC preceptors responded; 44% of preceptors (45/102) completed both block and LIC surveys. In total, 62% (68/110) of block clerkship and 83% (19/23) of LIC students responded. Both preceptors and students favoured evaluation in the LIC on three factors (p ≤ 0.01): validity of evaluation process, quality of clinical skill evaluation, and willingness to provide constructive feedback.

CONCLUSIONS

Preceptors and students perceived evaluation in an LIC more favourably than evaluation on block clerkships. For educators working to improve student evaluation, further examination of the LIC structure and evaluation processes that seem to enhance both formative assessment and summative evaluation may be useful to improve the quality of evaluation and feedback.

How Do Internal Medicine Residency Programs Evaluate Their Resident Float Experiences?

OBJECTIVES

We surveyed the nation's internal medicine residency training program directors to determine the range and frequency of existing methods by which float experiences are evaluated.

METHODS

We sent questionnaires to the program directors of all 396 internal medicine residency training program sites in the country. Information requested included program characteristics, months devoted to float experiences in each year of training, and the location and purpose of the rotation. Program directors were also asked to choose among descriptors characterizing the evaluative process.

RESULTS

There were 139 responding programs (39%), 134 with data that could be aggregated. Responding programs were similar to all programs nationally in the distributions of size and university sponsorship. Overall, 76% of programs employed a night float for any period of time, and 71% currently had one, on average for 6.7 years. Mean months of float experience during residency was 2.4 months, significantly longer in programs that were not university based. Float experiences were evaluated in 89% of those programs who employed them, with ten different methods reported. University-based programs were significantly less likely to use chart review as a method of evaluation, but no other differences in methodology were significant.

CONCLUSIONS

Float rotations are common among internal medicine residency training programs. Evaluative methods vary, but one or more are applied in the vast majority of programs.

What do faculty observe of medical students' clinical performance?

BACKGROUND

An earlier study of our faculty's evaluation of junior medical students indicated that performance ratings were unreliable and reflected 1 underlying dimension. Other researchers have obtained similar results.

PURPOSE

The purpose of this study was to identify which aspects of students' clinical performance faculty actually observe.

METHODS

We analyzed the responses of 9 faculty members to an open-ended questionnaire concerning which aspects of clinical performance attending faculty observe. We also reviewed and summarized the written comments of 331 faculty evaluations of third-year medical students.

RESULTS

Analysis of the questionnaires and evaluations indicated that faculty members gauge medical knowledge, professionalism, and clinical reasoning skills from direct interaction with students. History-taking and physical examination skills are inferred from the quality of verbal presentations. Faculty have little basis for evaluating other important aspects of clinical performance.

CONCLUSIONS

Faculty primarily observe medical students' cognitive skills and professionalism. Faculty have little basis for evaluating most other features of clinical performance.