Performance on the radiology component of a critical clinical skills examination

Student Evaluations, Outcomes, and National Licensure Examinations in Radiology Education: A Narrative Review of the Literature

OBJECTIVE

The purpose of this study was to examine literature on radiological student evaluation and outcome assessments including national board examinations.

METHODS

A review of the literature was performed using relevant key words. Articles were retrieved through December 2012 using PubMed, ScienceDirect, ERIC, Proquest, and ICL databases along with a manual review of references.

RESULTS

Of the 4716 unique abstracts reviewed by the author, 54 were found to be relevant to the purpose of this study. Student grade point average correlates with board scores in the nursing, chiropractic, and medical professions. Scores on the chiropractic college admission test and undergraduate grade point average correlate with success in professional college. There is a correlation between board scores and college attended. Board preparation programs do not appear to affect board examination scores.

CONCLUSION

Although evaluations can be effective teaching tools, they are not used by many radiology programs. Some programs have inadequate evaluations and do not allow students to review their evaluations. There are no definitive links between mastery of radiology and specific evaluations, outcomes, or pre-professional/clinical grades. Studies suggest that board examination scores reflect long-term mastery of knowledge rather than short-term memorization of facts.

Limitations of student evaluations of curriculum

RATIONALE AND OBJECTIVES

Medical student surveys are used extensively in the development and modification of curriculum. The purpose of this study was to look at medical student surveys of a radiology lecture series, evaluating the accuracy of student perceptions of learning and factors affecting them.

MATERIALS AND METHODS

After a "Case of the Week" lecture series, 156 3rd-year medical students returned a survey evaluating the experience with 10 questions on a four-point scale (1 = disagree, 4 = agree very much) and took a clinical competency assessment (CCA) examination with a radiology substation. Survey responses were compared with actual examination performance, analyzed for how overall learning was characterized in specific educational objectives, and evaluated for factors affecting perceived learning.

RESULTS

The mean response for perceived CCA examination preparedness was 1.83. The mean radiology station test score was 90.43%. Correlations between student perception of learning and the scoring of focused learning objectives ranged from 0.33 to 0.48 (P < .01). Students responding 1 to items assessing perceived lecture organization, stimulation to read, and interest in the field of radiology had mean scores for perception of overall learning of 2.09-2.44 and mean scores for recommendation of course continuation of 1.68-2.46. Students responding 4 had means of 3.25-3.81 and 3.06-4.0, respectively.

CONCLUSION

Student perceptions of the value of curriculum were inaccurate compared with external measures of performance, and students poorly related their general impressions to specific learning objectives. Perceived lecture organization, stimulation to read, and interest in radiology as a specialty affected perceived overall learning and perceived value of the lecture series.

Value of case-of-the-week presentations on the radiology component of the Comprehensive Clinical Assessment examination

RATIONALE AND OBJECTIVES

The Comprehensive Clinical Assessment (CCA) examination at the University of Michigan Medical School is a series of test stations through which the mastery of clinical skills is evaluated. The purpose of this study was to determine whether student performance on the radiology station improved in years when radiology faculty presented case-of-the-week unknowns to the 3rd-year students.

MATERIALS AND METHODS

The authors compared four separate classes of medical students in examination years 1993, 1994, 1995, and 1996 by using the total CCA examination score, the radiology station score, and radiology station pass/fail rates. Radiology case-of-the-week presentations were given by the radiology faculty only in academic years 1993-1994 and 1994-1995 (examination years 1994 and 1995).

RESULTS

The means and standard deviations of the radiology station scores for the examination years when case-of-the-week presentations were not given, 1993 and 1996, were 78.92 +/- 13.62 and 79.76 +/- 13.62, respectively. In the years case-of-the-week presentations were given, 1994 and 1995, the radiology station scores averaged 90.83 +/- 8.58 and 89.97 +/- 9.66, respectively (P < .001, global alpha = .05). Total CCA percentage correct scores were similar for all years studied. In 1993 and 1996, 7.6% and 5.3% of students, respectively, failed the radiology station. In 1994 and 1995, 0.4% and 0% of students, respectively, failed (P < .0001).

CONCLUSION

Case-of-the-week presentations by radiology faculty increased 3rd-year students' basic radiologic knowledge as evidenced by increased scores on the radiology station of the CCA examination.