Promoting the use of evidence-based medicine in pediatrics

A cluster quasi-randomized controlled trial of an interactive, monthly obstetric anesthesiology curriculum: impact on resident satisfaction and knowledge retention

BACKGROUND

Increasingly, evidence supports the use of educational paradigms that focus on teacher-learner interaction and learner engagement. We redesigned our monthly obstetric anesthesia resident didactics from a lecture-based curriculum to an interactive format including problem-based learning, case discussion, question/answer sessions, and simulation. We hypothesized that the new curriculum would improve resident satisfaction with the educational experience, satisfaction with the rotation, and knowledge retention.

METHODS

Fifty-three anesthesiology residents were prospectively recruited and quasi-randomized through an alternating-month pattern to attend either interactive sessions or traditional lectures. Residents completed a daily satisfaction survey about quality of teaching sessions and a comprehensive satisfaction survey at the conclusion of the rotation. Knowledge retention was assessed with a knowledge test completed on the final day. The primary outcome was daily satisfaction with the curriculum, and secondary outcomes included overall satisfaction with the curriculum, overall rotation satisfaction, and within-resident difference between pre- and post-knowledge test scores.

RESULTS

No differences were observed in daily resident satisfaction after interactive sessions vs traditional lectures. Furthermore, no differences were observed between the interactive sessions and traditional lecture groups in overall satisfaction with the curriculum, overall satisfaction with the entire rotation or within-resident difference between pre- and post-knowledge test scores.

CONCLUSIONS

Our study failed to demonstrate improvement in resident satisfaction or knowledge retention following implementation of an interactive curriculum on a month-long obstetric anesthesia rotation. Reasons may include misalignment of the intervention with measured study outcomes, lack of sensitivity of the survey tools, and inadequate training of faculty presenters.

Validity of the Health Systems Science Examination: Relationship Between Examinee Performance and Time of Training

The health systems science (HSS) framework articulates systems-relevant topics that medical trainees must learn to be prepared for physician practice. As new HSS-related curricula are developed, measures demonstrating appropriate levels of reliability and validity are needed. The authors describe a collaborative effort between a consortium of medical schools and the National Board of Medical Examiners to create a multiple-choice HSS examination in the areas of evidence-based medicine/population health, patient safety, quality improvement, and teamwork. Fifteen schools administered the 100-question examination through 2 academic years a total of 1887 times to 1837 first-time takers. Total test score mean was 67% (SD 11%). Total test reliability as measured by coefficient α was .83. This examination differentiated between medical students who completed the examination before, during, and after relevant training/instruction. This new HSS examination can support and inform the efforts of institutions as they integrate HSS-related content into their curricula.

A pediatrics-based instrument for assessing resident education in evidence-based practice

OBJECTIVE

The principles of evidence-based practice (EBP) are a mandated component of the pediatric residency curriculum; however, a pediatrics-based assessment tool validated with pediatric residents does not exist.

METHODS

We designed an assessment instrument composed of items in 4 categories: 1) demographics; 2) comfort level; 3) self-reported practice of EBP; and 4) EBP knowledge. This last section required participants to identify best evidence and most appropriate study design by using pediatric-based scenarios, develop searchable questions, and use existing published research to address diagnostic and treatment issues. Four groups completed the instrument: preclinical medical students (MS-2), incoming pediatric interns (PGY-1), incoming second- and third-year pediatric residents (PGY2-3), and expert tutors (expert). We determined internal consistency, interrater reliability, content validity, item difficulty, and construct validity.

RESULTS

Fifty-six subjects completed tests (MS-2, n = 13; PGY-1, n = 13; PGY2-3, n = 22; expert, n = 8). Internal reliability was good, with Cronbach's alpha = .80. Interrater reliability was high (kappa = 0.94). Items were free of floor or ceiling effects. Comfort level and self-reported practice of EBP increased with expertise level and prior EBP experience (P < .01). Scores on the knowledge section (out of 50 +/- SD) rose with training level (MS-2: 14.8 +/- 5.7; PGY-1: 22.2 +/- 3.4; PGY2-3: 31.7 +/- 6.1; experts: 43 +/- 4.0; P < .01). Scores also correlated with prior EBP education.

CONCLUSIONS

We have developed a reliable and valid instrument to assess knowledge and skill in EBP taught to pediatric residents. This instrument can aid pediatric educators in monitoring the impact of the EBP curriculum.