Evaluation of residents, faculty, and program

Do Residency Selection Factors Predict Radiology Resident Performance?

RATIONALE AND OBJECTIVES

The purpose of our study is to determine what information in medical student residency applications predicts radiology residency success as defined by objective clinical performance data.

MATERIALS AND METHODS

We performed a retrospective cohort study of residents who entered our institution's residency program through the National Resident Matching Program as postgraduate year 2 residents and completed the program over the past 2 years. Medical school grades, selection to Alpha Omega Alpha (AOA) Honor Society, United States Medical Licensing Examination (USMLE) scores, publication in peer-reviewed journals, and whether the applicant was from a peer institution were the variables examined. Clinical performance was determined by calculating each resident's cumulative major discordance rate for on-call cases the resident read and gave a preliminary interpretation. A major discordance was defined as a difference between the preliminary resident and the final attending interpretations that could immediately impact the care of the patient. A multivariate logistic regression was performed to determine significant variables.

RESULTS

Twenty-seven residents provided preliminary reports on call for 67,145 studies. The mean major discordance rate was 1.08% (range 0.34%-2.54%). Higher USMLE Step 1 scores, publication before residency, and election to AOA Honor Society were all statistically significant predictors of lower major discordance rates (P values 0.01, 0.01,  and <0.001, respectively).

CONCLUSIONS

Overall resident performance was excellent. There are predictors that help select the better performing residents, namely higher USMLE Step 1 scores, one to two publications during medical school, and election to AOA in the junior year of medical school.

Remedial training for the radiology resident: a template for optimization of the learning plan

All radiology residency programs should strive for the early identification of individuals in need of remedial training and have an approach ready to address this situation. This article provides a template for a step-by-step approach which is team based. It includes definition of the learning or performance issues, creation of suitable learning objectives and learning plan, facilitation of feedback and assessment, and definition of outcomes. Using such a template will assist the resident in returning to the path toward a safe and competent radiologist.

Impact of Electronic Resident Evaluation Systems on the Timeliness, Rate, and Quality of Responses

OBJECTIVE

To determine the impact of electronic resident evaluation systems on the timeliness, rate, and quality of responses.

METHODS

Surveys were mailed electronically to the membership of the Association of Program Directors in Radiology, which included the directors of 158 residency programs. Respondents were instructed to send 1 response from each program. Information gathered included the use of electronic compared with paper-based evaluation systems and the overall level of satisfaction with such systems (rated on a 5-point scale with 1 = dissatisfied and 5 = satisfied). Questions were asked regarding the impact of electronic systems on the timeliness, rate, and quality of responses.

RESULTS

Seventy-seven responses were received, for an estimated response rate of 49% on a per-program basis. Of these 77 respondents, 45 (58%) used electronic systems and 32 (42%) used paper-based systems. The median level of satisfaction was equivalent (4 = somewhat satisfied) for both groups. Of the 45 respondents using electronic systems, 26 (58%) reported increased response rates, 16 (36%) reported no change, and 3 (7%) reported decreased response rates compared with paper-based systems. Regarding the timeliness of responses, 31 (69%) reported faster response rates, 10 (22%) reported no change, and 4 (9%) reported slower rates compared with paper-based systems. Regarding the quality of responses, 25 (56%) reported no change, 12 (27%) reported improved quality, and 8 (18%) reported worse quality compared with paper-based systems.

CONCLUSION

Electronic systems are generally associated with an improved response rate and enhanced timeliness of responses in comparison with paper-based systems, without adversely affecting the quality of responses.

Mock radiology oral examination

RATIONALE AND OBJECTIVE

To organize and implement a mock oral examination for radiology residents in an attempt to prepare them for the American Board of Radiology Oral Examination.

MATERIALS AND METHODS

A mock oral examination was administered to junior and senior radiology residents by radiology faculty. All participants completed postexamination surveys.

RESULTS

The mock oral examination process not only provided practice for preboard residents, but also determined areas of deficiency in their fund of knowledge and presenting skills. Additionally, it provided faculty members with areas of curricular weakness.

CONCLUSION

Administration of a mock oral examination reaps multiple benefits, which far exceed the task of its implementation.

Evaluation of an online faculty appraisal instrument: comparison of resident and faculty perceptions

RATIONALE AND OBJECTIVES

Our purpose was to compare faculty and resident perceptions regarding an online faculty appraisal instrument and the potential impact of the appraisal process.

MATERIALS AND METHODS

Faculty members and residents at an academic medical center diagnostic radiology department were asked to complete anonymous surveys that sought feedback regarding an online faculty appraisal form and process. Questions were asked regarding the relative importance of various faculty performance measures, preferences for narrative versus quantitative components of the instrument, and the likely impact of the evaluation process on future faculty behavior.

RESULTS

The survey was completed by 19 (45%) of 42 faculty members and by 16 (80%) of 20 residents who participated in the initial faculty appraisal process. Residents and faculty both assigned generally high rankings to the six measures of faculty performance. The vast majority (86 %) of faculty and residents found the narrative component of the survey (either alone or in combination with quantitative data) most helpful. Based on the appraisal process, 74% of the faculty planned to modify one or two aspects of their behavior, whereas a significant minority (44%) of residents thought that the faculty would not make any changes.

CONCLUSION

Faculty and residents generally agree on a core set of faculty performance measures, and both groups show a preference for an appraisal instrument that incorporates a narrative component. Concerning the perceived impact of the appraisal process, faculty members were slightly more optimistic than residents regarding its ability to serve as an impetus for faculty behavioral changes.