Development of a radiology faculty appraisal instrument by using critical incident interviewing

Evaluation of an online faculty appraisal instrument: comparison of resident and faculty perceptions

RATIONALE AND OBJECTIVES

Our purpose was to compare faculty and resident perceptions regarding an online faculty appraisal instrument and the potential impact of the appraisal process.

MATERIALS AND METHODS

Faculty members and residents at an academic medical center diagnostic radiology department were asked to complete anonymous surveys that sought feedback regarding an online faculty appraisal form and process. Questions were asked regarding the relative importance of various faculty performance measures, preferences for narrative versus quantitative components of the instrument, and the likely impact of the evaluation process on future faculty behavior.

RESULTS

The survey was completed by 19 (45%) of 42 faculty members and by 16 (80%) of 20 residents who participated in the initial faculty appraisal process. Residents and faculty both assigned generally high rankings to the six measures of faculty performance. The vast majority (86 %) of faculty and residents found the narrative component of the survey (either alone or in combination with quantitative data) most helpful. Based on the appraisal process, 74% of the faculty planned to modify one or two aspects of their behavior, whereas a significant minority (44%) of residents thought that the faculty would not make any changes.

CONCLUSION

Faculty and residents generally agree on a core set of faculty performance measures, and both groups show a preference for an appraisal instrument that incorporates a narrative component. Concerning the perceived impact of the appraisal process, faculty members were slightly more optimistic than residents regarding its ability to serve as an impetus for faculty behavioral changes.

Curriculum in radiology for residents: what, why, how, when, and where

Developing a curriculum in chest radiology should follow the same general principles that are used when developing a curriculum in any subspecialty area of radiology. A curriculum is more than a "list of topics" with which a resident should be familiar after 4 years of training. It includes objectives and goals, content, faculty, methods, and evaluation. Numerous resources are available for those who are charged with developing a curriculum in chest radiology. In addition to faculty members in the department, whose input during development can ensure successful implementation of the curriculum, organizations (i.e., ACR, APDR, STR) already have begun to develop "model" curricula. Attending the annual meeting of the Association of American Medical Colleges is a way to meet and hear from professionals who develop and oversee curriculum development at their medical schools, and another important resource available at some medical schools is the Office of Medical Education. The faculty within such offices are uniquely qualified to assist with curriculum and faculty development, especially for those areas in which radiology faculty traditionally are less experienced, such as development of valid and reliable assessment forms and construction of behaviorally based objectives.

Radiology faculty evaluation: bi-institutional implementation of an experimental appraisal instrument

RATIONALE AND OBJECTIVES

The authors evaluated the reliability and validity of an experimental radiology faculty appraisal instrument.

MATERIALS AND METHODS

Residents from the University of Wisconsin (UW) and Indiana University (IU) were asked to use a previously developed behaviorally based, 53-item experimental faculty appraisal instrument. Twenty UW residents evaluated 29 UW faculty members and 37 IU residents evaluated 31 IU faculty members by using the experimental instrument. Residents also evaluated faculty by using their institution's existing appraisal instrument.

RESULTS

Correlations between existing and experimental forms were .69 and .87 for UW and IU, respectively. Existing form reliabilities were .89 and .94 and experimental form reliabilities were .98 and .98 for UW and IU, respectively. Experimental form length was reduced to 30 items by eliminating the questions that correlated the least with section scores. Reliabilities of scores on the shortened form were .97 and .98 and correlated .65 and .88 with scores on the longer form for UW and IU, respectively.

CONCLUSION

Ratings obtained with the existing forms correlated substantially with the experimental form, attesting to the experimental form's validity. Shortening the experimental form had a minimal effect on the reliability and validity of the data obtained. The behavior-based form was used to rate behaviors that residents believed discriminated between effective and ineffective instructors, enabling an objective and relevant assessment to be made.