Abstract
STUDY OBJECTIVE
To assess professionalism in anesthesiology residents, it is important to obtain evaluations from people with whom they interact on daily basis. The purpose of this study was to evaluate the effect of a Multisource feedback (MSF) on resident's professional behavior and to assess the effect of faculty feedback on resident performance.
DESIGN
This study was a two-group randomized clinical trial.
SETTING
Residents were recruited from Cleveland Clinic Children's Hospital.
PATIENTS
Participants included twenty eight residents doing a two-month rotation in Pediatric Anesthesia.
INTERVENTIONS
Multisource feedback questionnaires were developed and then validated using face and content validity. Residents were randomly assigned to a feedback group or a control group. Both groups received the MSF evaluation. Only the group assigned to feedback had a 'coaching meeting' every month creating strategies for improvement.
MEASUREMENTS
MSF questionnaires were validated using a face validation and expert content validity. The effect of MSF on a professionalism questionnaire was assessed using analysis of covariance and linear mixed effects regression models.
MAIN RESULTS
Observed test-retest agreement was greater than 0.90 for all items, with more than half of kappa statistics greater than 0.50. Cronbach's alpha was 0.71.The MSF increased the self-assessment score with an estimated effect of 0.21 (95% CI 0.06, 0.37), P=.015. There was no detected effect on patient family evaluation, with mean difference (CI) in change from baseline of 0.03 (-0.15, 0.21), P=.77, faculty evaluation, 0.21 (-0.02, 0.44), P=.08, or coworker evaluation 0.13 (-0.11, 0.37).
CONCLUSIONS
Our new multi-source feedback questionnaire to assess professionalism had good reliability and internal consistency. Using our validated questionnaire we assessed the effect of a monthly feedback to improve professionalism in anesthesia residents. While we did see improvement in anesthesiology residents' self-assessment, we did not see a similar effect on patient family, faculty or coworker evaluations.
Collapse