The #PedsMatch21 Webinar Series: Coordinated Specialty-Level Communication During the Virtual Residency Application Cycle

PROBLEM

The landscape of pediatric medical education changed significantly in the wake of the COVID-19 pandemic. Fourth-year medical students applying to pediatric residency programs were particularly affected by alterations to traditional away rotations, the inability to conduct in-person visits, and resulting changes in ways of communicating with residency programs. Applicants were concerned about the virtual component of the application cycle and worried as to how to manage the COVID-19-related changes. Program leaders also faced apprehension at how to close communication gaps and effectively use virtual tools to reach applicants.

APPROACH

Founders of the Twitter-based @FuturePedsRes (FPR) organization recognized these concerns and developed a webinar series during the 2020-2021 academic year to create opportunities for programs and applicants to communicate. Leaders of the Association of Pediatric Program Directors and the Council on Medical Student Education in Pediatrics joined as sponsors and co-hosts. The #PedsMatch21 webinar series consisted of 9 total webinars: 2 overviews, 5 regional, 1 for osteopathic students, and 1 for international medical graduates.

OUTCOMES

A total of 138 pediatric residency programs across the country participated in the #PedsMatch21 webinar series. Each webinar brought in a mean of 431 attendees, from the United States and several other countries. Follow-up surveys immediately after each webinar and again at the end of the interview season demonstrated that the webinars decreased applicants' anxiety, provided helpful information on the virtual season, and increased their awareness of programs.

NEXT STEPS

The #PedsMatch21 webinar series demonstrated utility in addressing applicants' concerns and allowing applicants access to a greater number of programs. Webinar-based strategies should be considered across all medical specialties as a useful method of providing accessible forums for programs and applicants.

The Impact of the COVID-19 Pandemic on Pediatric Graduate Medical Education: Lessons Learned and Pathways Forward

In this article, the authors describe the impact of the COVID-19 pandemic on pediatric graduate medical education (GME), including the impact on clinical experiences for trainees, teaching methods used, trainee wellness, GME leader wellness and support, and the traditional interview process. A thorough literature review was done to identify impacts of the COVID-19 pandemic on pediatric GME. In addition, information was collected through Association of Pediatric Program Directors virtual cafes and conferences. Positive changes for GME from the COVID-19 pandemic included: the rapid transition to telehealth; asynchronous learning allowing for increased cross-program collaboration; innovative online teaching modalities; increased flexibility and decreased cost of online recruitment; and shared innovations across pediatric GME. Challenging aspects of the COVID-19 pandemic included: decreased learning about common childhood illnesses, such as bronchiolitis, acute otitis media, and influenza; decreased patient volumes and patient complexity in clinics and inpatient wards, leading to less practice developing efficiency, time management, and triaging skills; and an increased burden on trainees, including moral distress and decreased support from one another and other social supports. The COVID-19 pandemic has highlighted important opportunities in U.S. educational systems. As medical educators move forward, it will be important to learn from these while mitigating the negative impacts.

National Pediatric Experience With Virtual Interviews: Lessons Learned and Future Recommendations

The coronavirus disease 2019 pandemic significantly impacted undergraduate and graduate medical education and created challenges that prevented a traditional approach to residency and fellowship recruitment and interviews. Early in the pandemic, the pediatric education community came together to support applicants and training programs and to foster an equitable recruitment process. We describe many of our community's innovations, including the use of virtual cafés to educate programs and highlight best practices for virtual recruitment and the use of regional webinars to highlight residency programs and provide information to applicants. Surveys of applicants and programs suggest that the virtual interview process worked well overall, with applicants and programs saving both time and money and programs maintaining a high rate of filling their positions. On the basis of this experience, we highlight the strengths and weaknesses of 3 potential models for future interview seasons. We close with a series of questions that need further investigation to create an effective and equitable recruitment process for the future.

The Development, Implementation, and Evaluation of an Acute Otitis Media Education Website

OBJECTIVE

To develop, implement, and evaluate an acute otitis media (AOM) education website for clinician-educators.

METHODS

We developed an education website following Kern's curriculum model.

RESULTS

The website contained peer-reviewed content, educational objectives, library search pages to identify evidence-based resources, and a faculty toolbox with instructional and evaluation instruments. Pediatric clinician-educators were purposefully sampled from different clinic sites to evaluate the website. Semistructured interviews explored key website components for content and usability in clinical teaching. In grounded theory tradition, investigators used the constant comparative method with qualitative analysis software to identify themes and representative quotations. Eleven faculty members (9 females and 2 males with teaching experience from 6 to 26 years) participated in the study. Identified themes were: 1) value of visual impact for learning specific topics, 2) promotion of efficiency in teaching clinical topics, 3) varying approaches for using website, and 4) faculty's self-report of knowledge and self-efficacy needs.

CONCLUSIONS

An education website may enhance the teaching of AOM, accommodate different teaching preferences, promote efficiency in teaching, and advance clinician-educator knowledge and skills. Next steps include evaluation of learners' perspectives, generalizability in varied teaching settings, and assessment of higher learning outcomes including impact on knowledge, skills, and patient outcomes.

Linguistic analysis of pediatric residency personal statements: gender differences

BACKGROUND

All US residency programs require applicants to submit personal statements. Prior studies showed gender differences in personal statement writing, which has implications for gender bias in the application process, but previous studies have not considered the dual influence of specialty-specific values on personal statement writing by applicants of each gender.

OBJECTIVE

To understand gender differences in pediatric residency personal statements.

METHODS

From 2017 to 2018, we performed linguistic analysis of personal statements written by interviewees at a mid-size US pediatrics residency during two prior academic years. We assessed writing tone, communal language, and agentic language. We performed t-tests to evaluate for gender differences, p < 0.05.

RESULTS

We analyzed personal statements from 85 male and 85 female interviewees. Average word count was 676 words. Personal statements demonstrated analytic writing style with authentic and positive emotional tone. We found no gender differences in communal language for social affiliation (p = 0.31), adjectives (p = 0.49), or orientation (p = 0.48), which deviates from typical gender norms for male language use. Males used agentic language of reward more frequently (p = 0.02).

CONCLUSIONS

Findings suggest that social language is valued in pediatrics, a predominantly female specialty, regardless of applicant gender. Use of reward language by males is consistent with previous findings. Future studies should evaluate gender differences in residency applications across specialties to advance understanding of the role gender plays in the application process.

Key Factors in Clinical Competency Committee Members' Decisions Regarding Residents' Readiness to Serve as Supervisors: A National Study

PURPOSE

Entrustment has become a popular assessment framework in recent years. Most research in this area has focused on how frontline assessors determine when a learner can be entrusted. However, less work has focused on how these entrustment decisions are made. The authors sought to understand the key factors that pediatric residency program clinical competency committee (CCC) members consider when recommending residents to a supervisory role.

METHOD

CCC members at 14 pediatric residency programs recommended residents to one of five progressive supervisory roles (from not serving as a supervisory resident to serving as a supervisory resident in all settings). They then responded to a free-text prompt, describing the key factors that led them to that decision. The authors analyzed these responses, by role recommendation, using a thematic analysis.

RESULTS

Of the 155 CCC members at the participating programs, 84 completed 769 supervisory role recommendations during the 2015-2016 academic year. Four themes emerged from the thematic analysis: (1) Determining supervisory ability follows from demonstrated trustworthiness; (2) demonstrated performance matters, but so does experience; (3) ability to lead a team is considered; and (4) contextual considerations external to the resident are at play.

CONCLUSIONS

CCC members considered resident and environmental factors in their summative entrustment decision making. The interplay between these factors should be considered as CCC processes are optimized and studied further.

Teaching Pediatric Otoscopy Skills to Pediatric and Emergency Medicine Residents: A Cross-Institutional Study

OBJECTIVE

To evaluate a pediatric otoscopy curriculum with the use of outcome measures that included assessment of skills with real patients.

METHODS

Thirty-three residents in an intervention group from 2 institutions received the curriculum. In the previous year, 21 residents in a nonintervention group did not receive the curriculum. Both groups were evaluated at the beginning and end of their internship years with the use of the same outcome assessments: 1) a written test, 2) an objective standardized clinical examination (OSCE), and 3) direct observation of skills in real patients with the use of a checklist with established validity.

RESULTS

The intervention group had a significant increase in percentage reaching minimum passing levels between the beginning and end of the internship year for the written test (12% vs 97%; P < .001), OSCE (0% vs 78%; P < .001), and direct observation (0% vs 75%; P < .001); significant mean percentage gains for the written test (21%; P < .001), OSCE (28%; P < .001), and direct observation (52%; P = .008); and significantly higher (P < .001) mean percentage gains than the nonintervention group on the written test, OSCE, and direct observation. The nonintervention group did not have a significant increase (P = .99) in percentage reaching minimum passing levels, no significant mean percentage gains in the written test (2.7%; P = .30) and direct observation (6.7%; P = .61), and significant regression in OSCE (-5.2%; P = .03).

CONCLUSIONS

A pediatric otoscopy curriculum with multimodal outcome assessments was successfully implemented across different specialties at multiple institutions and found to yield gains, including in skills with real patients.

Thresholds and interpretations: How clinical competency committees identify pediatric residents with performance concerns

BACKGROUND

Clinical competency committee (CCC) identification of residents with performance concerns is critical for early intervention.

METHODS

Program directors and 94 CCC members at 14 pediatric residency programs responded to a written survey prompt asking them to describe how they identify residents with performance concerns. Data was analyzed using thematic analysis.

RESULTS

Six themes emerged from analysis and were grouped into two domains. The first domain included four themes, each describing a path through which residents could meet or exceed a concern threshold:1) written comments from rotation assessments are foundational in identifying residents with performance concerns, 2) concerning performance extremes stand out, 3) isolated data points may accumulate to raise concern, and 4) developmental trajectory matters. The second domain focused on how CCC members and program directors interpret data to make decisions about residents with concerns and contained 2 themes: 1) using norm- and/or criterion-referenced interpretation, and 2) assessing the quality of the data that is reviewed.

CONCLUSIONS

Identifying residents with performance concerns is important for their education and the care they provide. This study delineates strategies used by CCC members across several programs for identifying these residents, which may be helpful for other CCCs to consider in their efforts.

Using educational prescriptions to teach medical students evidence-based medicine

PURPOSE

To evaluate feasibility and impact of evidence-based medicine (EBM) educational prescriptions (EPs) in medical student clerkships.

METHODS

Students answered clinical questions during clerkships using EPs, which guide learners through the "four As" of EBM. Epidemiology fellows graded EPs using a rubric. Feasibility was assessed using descriptive statistics and student and fellow end-of-study questionnaires, which also measured impact. In addition, for each EP, students reported patient impact. Impact on EBM skills was assessed by change in EP scores over time and scores on an EBM objective structured clinical exam (OSCE) that were compared to controls from the prior year.

RESULTS

117 students completed 402 EPs evaluated by 24 fellows. Average score was 7.34/9.00 (SD 1.58). 69 students (59%) and 21 fellows (88%) completed questionnaires. Most students thought EPs improved "Acquiring" and "Appraising". Almost half thought EPs improved "Asking" and "Applying". Fellows did not value grading EPs. For 18% of EPs, students reported a "change" or "potential change" in treatment. 56% "confirmed" treatment. EP scores increased by 1.27 (95% CI: 0.81-1.72). There were no differences in OSCE scores between cohorts.

CONCLUSIONS

Integrating EPs into clerkships is feasible and has impact, yet OSCEs were unchanged, and research fellows had limitations as evaluators.

Standardized Checklist for Otoscopy Performance Evaluation: A Validation Study of a Tool to Assess Pediatric Otoscopy Skills

INTRODUCTION

Acute otitis media (AOM) is the most frequently diagnosed pediatric illness in the United States and the most common indication for antibiotic prescription. Skill in pediatric otoscopy is essential to correctly identify children with AOM. However, studies have found diagnostic inconsistency and significant overdiagnosis among practitioners. Given the potential public and individual health consequences, there has been a call for improved education regarding the diagnostic certainty of AOM. Yet educational efforts continue to be limited, particularly in regard to competency assessment. The lack of a validated tool to assess otoscopy skill attainment objectively diminishes the instructor's ability to provide useful feedback and direction to the learner.

METHODS

We have undertaken an educational intervention with the goal of developing a validated Standardized Checklist for Otoscopy Performance Evaluation (SCOPE), building on key principles of the general pediatric ear exam. The SCOPE was developed with the input of process and content experts with attention to specific domains of validity.

RESULTS

Our analysis provides important validity evidence for the SCOPE assessment tool. The instrument was piloted and successfully implemented with medical students and varying levels of residents in pediatrics and emergency medicine over a 5-year period in varied settings: urgent care, large and small pediatric clinics, and the emergency departments at two institutions. It has been used for both instruction and assessment purposes.

DISCUSSION

Because the SCOPE can be used in teaching demonstration purposes, in formative and summative assessment settings, and across the continuum of learners, this instrument offers the potential for more educational efforts in the field of assessment in direct patient care. We anticipate that the SCOPE will foster an environment of efficient yet high-yield review and discussion of otoscopy and diagnostic competency.

Patient safety and quality improvement: a 'CLER' time to move beyond peripheral participation

In the United States, the Accreditation Council for Graduate Medical Education (ACGME) has instituted a new program, the Clinical Learning Environment Review (CLER), that places focus in six important areas of the resident and fellow working and learning environment. Two of these areas are patient safety and quality improvement (QI). In their early CLER reviews of institutions housing ACGME-accredited training programs, ACGME has found that despite significant progress in patient safety and QI to date much work remains, especially when it comes to meaningful engagement of medical trainees in this work. In this article, the authors argue that peripheral involvement of trainees in patient safety and QI work does not allow the experiential learning that is necessary for professional development and the ultimate ability to execute performance that meets the needs of patients in contemporary clinical practice. Rather, as leaders in patient safety and QI have advocated since early in this movement, embedded and immersed experiences are necessary for learning and success.

The Power of Collaboration: Experiences From the Educational Innovations Project and Implications for the Next Accreditation System

BACKGROUND

The Internal Medicine Educational Innovations Project (EIP) is a 10-year pilot project for innovating in accreditation, which involves annual reporting of information and less-restrictive requirements for a group of high-performing programs. The EIP program directors' experiences offer insight into the benefits and challenges of innovative approaches to accreditation as the Accreditation Council for Graduate Medical Education transitions to the Next Accreditation System.

OBJECTIVE

We assessed participating program directors' perceptions of the EIP at the midpoint of the project's 10-year life span.

METHODS

We conducted telephone interviews with 15 of 18 current EIP programs (83% response rate) using a 19-item, open-ended, structured survey. Emerging themes were identified with content analysis.

RESULTS

Respondents identified a number of the benefits from the EIP, most prominent among them, collaboration between programs (87%, 13 of 15) and culture change around quality improvement (47%, 7 of 15). The greatest benefit for residents was training in quality improvement methods (53%, 8 of 15), enhancing those residents' ability to become change agents in their future careers. Although the requirement for annual data reporting was identified by 60% (9 of 15) of program directors as the biggest challenge, respondents also considered it an important element for achieving progress on innovations. Program directors unanimously reported their ability to sustain innovation projects beyond the 10-year participation in EIP.

CONCLUSIONS

The work of EIP was not viewed as "more work," but as "different work," which created a new mindset of continuous quality improvement in residency training. Lessons learned offer insight into the value of collaboration and opportunities to use accreditation to foster innovation.

Putting the pediatrics milestones into practice: a consensus roadmap and resource analysis

The Accreditation Council for Graduate Medical Education has partnered with member boards of the American Board of Medical Specialties to initiate the next steps in advancing competency-based assessment in residency programs. This initiative, known as the Milestone Project, is a paradigm shift from traditional assessment efforts and requires all pediatrics residency programs to report individual resident progression along a series of 4 to 5 developmental levels of performance, or milestones, for individual competencies every 6 months beginning in June 2014. The effort required to successfully make this shift is tremendous given the number of training programs, training institutions, and trainees. However, it holds great promise for achieving training outcomes that align with patient needs; developing a valid, reliable, and meaningful way to track residents' development; and providing trainees with a roadmap for learning. Recognizing the resources needed to implement this new system, the authors, all residency program leaders, provide their consensus view of the components necessary for implementing and sustaining this effort, including resource estimates for completing this work. The authors have identified 4 domains: (1) Program Review and Development of Stakeholders and Participants, (2) Assessment Methods and Validation, (3) Data and Assessment System Development, and (4) Summative Assessment and Feedback. This work can serve as a starting point and framework for collaboration with program, department, and institutional leaders to identify and garner necessary resources and plan for local and national efforts that will ensure successful transition to milestones-based assessment.

Academic medicine: a key partner in strengthening the primary care infrastructure via teaching health centers

In the United States, a worsening shortage of primary care physicians, along with structural deficiencies in their training, threaten the primary care system that is essential to ensuring access to high-quality, cost-effective health care. Community health centers (CHCs) are an underused resource that could facilitate rapid expansion of the primary care workforce and simultaneously prepare trainees for 21st-century practice. The Teaching Health Center Graduate Medical Education (THCGME) program, currently funded by the Affordable Care Act, uses CHCs as training sites for primary-care-focused graduate medical education (GME).The authors propose that the goals of the THCGME program could be amplified by fostering partnerships between CHCs and teaching hospitals (academic medical centers [AMCs]). AMCs would encourage their primary care residency programs to expand by establishing teaching health center (THC) tracks. Modifications to the current THCGME model, facilitated by formal CHC and academic medicine partnerships (CHAMPs), would address the primary care physician shortage, produce physicians prepared for 21st-century practice, expose trainees to interprofessional education in a multidisciplinary environment, and facilitate the rapid expansion of CHC capacity.To succeed, CHAMP THCs require a comprehensive consortium agreement designed to ensure equity between the community and academic partners; conforming with this agreement will provide the high-quality GME necessary to ensure residency accreditation. CHAMP THCs also require a federal mechanism to ensure stable, long-term funding. CHAMP THCs would develop in select CHCs that desire a partnership with AMCs and have capacity for providing a community-based setting for both GME and health services research.

PEARLs of Wisdom: Impact of a New Block Conference on Pediatrics Resident Attendance, Satisfaction, and Learning

BACKGROUND

Resident attendance and participation at didactic conferences is often limited owing to time demands. In 2010, University of Wisconsin-Madison pediatrics residency eliminated all noon conferences and implemented a new block format, PEARL (Pediatric Education and Active Resident Learning).

OBJECTIVE

The purpose of this study was to assess whether changes in a conference structure improved resident attendance, distractibility, satisfaction, perception of clinical relevance, and connection.

METHODS

Pediatrics residents were surveyed to compare experiences in 2 different conference structures: a traditional noon conference and an interactive case-based PEARL conference. Pediatrics residents from the 2008-2010 classes were surveyed about noon conference experiences and 2009-2011 classes were surveyed about PEARL conference experiences.

RESULTS

Participants included 32 residents in the 2010 presurvey and 36 in the 2011 postsurvey. All measures of attendance, distractibility, and satisfaction showed positive changes. For example, the average proportion of conferences attended was 73% with noon conferences and 98% with PEARL (P  =  .001). However, measures of perceived clinical relevance and resident participation did not change significantly. For example, on average 47% of residents reported contributing comments in a noon conference, whereas 56% of residents reported contributing in a block conference (P  =  .199).

CONCLUSIONS

PEARL conference significantly improved resident attendance, lowered distractibility, and improved resident satisfaction. However, these structural changes did not lead to changes in perceived clinical relevance of information learned or resident participation. Further content changes or faculty teaching strategies should be considered.

Validation of an Evidence-Based Medicine Critically Appraised Topic Presentation Evaluation Tool (EBM C-PET)

BACKGROUND

Competence in evidence-based medicine (EBM) is an important clinical skill. Pediatrics residents are expected to acquire competence in EBM during their education, yet few validated tools exist to assess residents' EBM skills.

OBJECTIVE

We sought to develop a reliable tool to evaluate residents' EBM skills in the critical appraisal of a research article, the development of a written EBM critically appraised topic (CAT) synopsis, and a presentation of the findings to colleagues.

METHODS

Instrument development used a modified Delphi technique. We defined the skills to be assessed while reviewing (1) a written CAT synopsis and (2) a resident's EBM presentation. We defined skill levels for each item using the Dreyfus and Dreyfus model of skill development and created behavioral anchors using a frame-of-reference training technique to describe performance for each skill level. We evaluated the assessment instrument's psychometric properties, including internal consistency and interrater reliability.

RESULTS

The EBM Critically Appraised Topic Presentation Evaluation Tool (EBM C-PET) is composed of 14 items that assess residents' EBM and global presentation skills. Resident presentations (N  =  27) and the corresponding written CAT synopses were evaluated using the EBM C-PET. The EBM C-PET had excellent internal consistency (Cronbach α  =  0.94). Intraclass correlation coefficients were used to assess interrater reliability. Intraclass correlation coefficients for individual items ranged from 0.31 to 0.74; the average intraclass correlation coefficients for the 14 items was 0.67.

CONCLUSIONS

We identified essential components of an assessment tool for an EBM CAT synopsis and presentation with excellent internal consistency and a good level of interrater reliability across 3 different institutions. The EBM C-PET is a reliable tool to document resident competence in higher-level EBM skills.

Enhancing teamwork between chief residents and residency program directors: description and outcomes of an experiential workshop

BACKGROUND

An effective working relationship between chief residents and residency program directors is critical to a residency program's success. Despite the importance of this relationship, few studies have explored the characteristics of an effective program director-chief resident partnership or how to facilitate collaboration between the 2 roles, which collectively are important to program quality and resident satisfaction. We describe the development and impact of a novel workshop that paired program directors with their incoming chief residents to facilitate improved partnerships.

METHODS

The Accreditation Council for Graduate Medical Education sponsored a full-day workshop for residency program directors and their incoming chief residents. Sessions focused on increased understanding of personality styles, using experiential learning, and open communication between chief residents and program directors, related to feedback and expectations of each other. Participants completed an anonymous survey immediately after the workshop and again 8 months later to assess its long-term impact.

RESULTS

Participants found the workshop to be a valuable experience, with comments revealing common themes. Program directors and chief residents expect each other to act as a role model for the residents, be approachable and available, and to be transparent and fair in their decision-making processes; both groups wanted feedback on performance and clear expectations from each other for roles and responsibilities; and both groups identified the need to be innovative and supportive of changes in the program. Respondents to the follow-up survey reported that workshop participation improved their relationships with their co-chiefs and program directors.

CONCLUSION

Participation in this experiential workshop improved the working relationships between chief residents and program directors. The themes that were identified can be used to foster communication between incoming chief residents and residency directors and to develop a curriculum for chief resident development.

A validated search assessment tool: assessing practice-based learning and improvement in a residency program

OBJECTIVE

The objective of this study was to validate an assessment instrument for MEDLINE search strategies at an academic medical center.

METHOD

Two approaches were used to investigate if the search assessment tool could capture performance differences in search strategy construction. First, data from an evaluation of MEDLINE searches from a pediatric resident's longitudinal assessment were investigated. Second, a cross-section of search strategies from residents in one incoming class was compared with strategies of residents graduating a year later. MEDLINE search strategies formulated by faculty who had been identified as having search expertise were used as a gold standard comparison. Participants were presented with a clinical scenario and asked to identify the search question and conduct a MEDLINE search. Two librarians rated the blinded search strategies.

RESULTS

Search strategy scores were significantly higher for residents who received training than the comparison group with no training. There was no significant difference in search strategy scores between senior residents who received training and faculty experts.

CONCLUSION

The results provide evidence for the validity of the instrument to evaluate MEDLINE search strategies. This assessment tool can measure improvements in information-seeking skills and provide data to fulfill Accreditation Council for Graduate Medical Education competencies.

Teaching primary care in community health centers: addressing the workforce crisis for the underserved

Universal coverage and multiple initiatives to improve health care delivery are crucial components of health care reform. However, the missing link has been a plan to rapidly address the primary care workforce crisis for the underserved. The authors propose a link between primary care graduate medical education and care for the underserved in community health centers, where expansion will be necessary for the anticipated increase in Medicaid and insured patients. This can be achieved by establishing primary care teaching health centers in expanded community health centers, which have established a patient-centered medical home practice environment. Residents would receive their final year of training in these centers, and then have the incentive of National Health Service Corps debt repayment if they subsequently practice in an underserved area. Primary care residents being trained in this setting would immediately increase the clinical capacity of community health centers and ultimately expand the primary care physician workforce. This proposal addresses the primary care physician workforce crisis and the associated key problems of limited access for the underserved and suboptimal primary care graduate medical education.

Choosing a career in combined internal medicine-pediatrics: insights from interns

BACKGROUND

Combined internal medicine-pediatrics (med-peds) programs may be competing for the same students who would have otherwise chosen family medicine. The degree to which this is happening is not known.

METHODS

We sent an eight-item questionnaire to new med-peds interns to assess their career plans at different stages of their decision making. Questionnaires were mailed to the directors of all US med-peds programs in 2002.

RESULTS

A total of 288/333 (87%) responded. The med-peds interns were more likely to be interested in internal medicine or pediatrics than they were in family medicine. If med-peds were not available, only 52/286 (18%) would have chosen family medicine as an alternative. A total of 55/288 (19%) anticipated practicing in rural areas.

CONCLUSIONS

The majority of med-peds interns would have chosen internal medicine or pediatrics if med-peds was not available. A small percentage would have chosen family medicine, thus having a minor impact on recruitment. An even smaller proportion would have chosen a non-primary care specialty. A sizable number anticipate practicing in rural areas.

Development of an evaluation of medical student competence in evidence-based medicine using a computer-based OSCE station

BACKGROUND

Instruction in evidence-based medicine (EBM) has been widely incorporated into medical school curricula with little evidence of its effectiveness. Our goal was to create, implement, and validate a computer-based assessment tool that measured medical students' EBM skills.

DESCRIPTION

As part of a required objective structured clinical examination, we developed a specific case scenario in which students (a) asked a structured clinical question using a standard framework, (b) generated effective MEDLINE search terms to answer a specific question, and (c) elected the most appropriate of 3 abstracts generated from a search justifying which best applies to the patient scenario.

EVALUATION

Between the 3 blinded raters, there was very good interrater reliability with 84, 94, and 96% agreement on the scoring for each component, respectively (k = .64, .82, and .91, respectively). In addition, students found the station appropriately difficult for their level of training.

CONCLUSIONS

This computer-based tool appears to measure several EBM skills independently and combines simple administration and scoring. Its generalizability to other cases and settings requires further study.

Assessing residents' competencies at baseline: identifying the gaps

PURPOSE

Entering residents have variable medical school experiences and differing knowledge and skill levels. To structure curricula, enhance patient safety, and begin to meet accreditation requirements, baseline assessment of individual resident's knowledge and skills is needed. To this end, in 2001 the University of Michigan Health System created the Postgraduate Orientation Assessment (POA), an eight-station, objective structured clinical examination for incoming residents.

METHOD

The POA, administered at orientation, included items addressing critical laboratory values, cross-cultural communication, evidence-based medicine, radio-graphic image interpretation, informed consent, pain assessment and management, aseptic technique, and system compliance such as fire safety. The POA assessed many of the skills needed by interns in their initial months of training when supervision by senior physicians might not be present.

RESULTS

In 2002, 132 interns from 14 different specialties and 59 different schools participated in the POA. The mean score was 74.8% (SD = 5.8). When scores were controlled for U.S. Medical Licensing Examination scores, there were no significant differences in performance across specialties. There were differences between University of Michigan Medical School graduates and those from other institutions (p <.001). Eighty-one percent of the residents would recommend the POA.

CONCLUSIONS

The POA provides a feasible format to measure initial knowledge and skills and identify learning needs. Orientation is an effective time to identify important gaps in learning between medical school and residency. This is the first step in a continuing evaluation of the Accreditation Council for Graduate Medical Education's general competencies.

Internal medicine-pediatrics residency training: current program trends and outcomes

PURPOSE

Combined internal medicine-pediatrics (med-peds) residency programs have existed since 1967. Due to the rapid growth in the number and size of programs during the 1990s, most current med-peds physicians completed their residency in the last ten years, making older studies of med-peds programs obsolete. The authors sought to determine completion rates of med-peds residency programs and describe the initial career plans for five cohorts of graduating residents from combined med-peds training programs.

METHOD

Program directors of all U.S. med-peds residency programs were asked to complete a Web-based survey and base their responses on the records of cohorts of residents completing their programs from 1998 through 2002. To allow sufficient time to complete both the American Board of Pediatrics (ABP) and American Board of Internal Medicine (ABIM) certification examinations, certification status was requested only for the cohort completing training in 1998.

RESULTS

Responses were obtained from 92% (83/90) of the programs, reflecting 1,595 residents entering med-peds programs. Of these residents, 91% graduated from a med-peds program. Among the graduates, 82% were seeing both adults and children, 22% went on to subspecialty residencies, 21% began practice in rural or underserved areas, and 25% entered an academic position. ABIM and ABP pass rates for the 1998 cohort were 97% and 96%, respectively. Overall, 79% of the 1998 graduates are board certified in both specialties.

CONCLUSIONS

Compared with previous studies, a greater proportion of residents who recently entered med-peds programs completed their dual training, and a larger percentage of graduates are seeing both adults and children. The proportion of residents entering subspecialty residencies has increased significantly, but the proportion of graduates in academic careers has remained stable.

Assessing residents' competency in care management: report of a consensus conference

BACKGROUND

Residency programs must prepare physicians to practice in the current health care environment. This mandate is reflected in 3 of the 6 competency domains now required by the Accreditation Council for Graduate Medical Education: systems-based practice, interpersonal skills and communication, and practice-based learning and improvement.

SUMMARY

An invitational conference was convened, with experts in clinical practice, managed care administration, and education to identify and recommend optimal and promising assessment methods for 4 target areas: physician-patient communication, ethics, teamwork and collaboration, and practice management. Working in small groups, participants considered a range of resident assessment methods and identified current or future methods for each area, based on reliability, validity, use of behaviorally oriented outcomes, feasibility, and cost. Preferred methods of assessment varied by domain and include written examinations, computer-based patient management problems, standardized patients, objective structured clinical examinations, portfolios, 360-degree evaluations, and patient satisfaction surveys.

CONCLUSIONS

The use of several practical, scientifically sound, and specific methods for assessing residents' competency in care management are recommended. Assessment instruments will need to be flexible enough to adapt to the rapid changes in the health care delivery system and terminology.

Impact of a veterans affairs continuity clinic on resident competencies in women's health

BACKGROUND

Education in women's health is now considered a core curricular component during residency training in Internal Medicine. There is potential for insufficient training in women's health for residents with a continuity clinic based at a Veterans Affairs (VA) hospital.

OBJECTIVE

To determine the impact of a 3-year continuity clinic based at a VA hospital on residents' self-reported competencies in women's health.

DESIGN

Cross sectional survey using an internal website.

SETTING

University-based residency program in Ann Arbor, Michigan.

MEASUREMENTS AND MAIN RESULTS

Comparison of residents with a VA clinic with residents with non-VA clinics (university and community) in self-reported competencies in knowledge base, counseling, and physical exam skills in the area of women's health. Responses were obtained from 66% (n = 72) of eligible residents. When compared to residents with either a university hospital- or community-based clinic site, VA-based residents reported less confidence in the majority of competencies surveyed. Clinic site had the strongest impact in the knowledge base domain, accounting for between 17% and 33% of the variance in each specific competency. For estimated number of Pap smears and breast exams done in the prior year, VA-based residents reported doing, on average, less than 5 of each per year while non-VA residents reported doing between 11 and 20 of each exam.

CONCLUSIONS

Our data suggest that despite other clinical opportunities in women's health during ambulatory rotations, regular clinical experiences in women's health in the continuity clinic setting are necessary to improve education in this area.

A computer-based OSCE station to measure competence in evidence-based medicine skills in medical students

OBJECTIVE

To create a feasible, valid, and reliable tool to measure third-year medical students' skills in evidence-based medicine (EBM).

DESCRIPTION

EBM skills-asking clinical questions, finding appropriate medical information resources, and appraising and applying them to patients-involve higher-order critical thinking abilities and are essential to being a competent physician. Students at our institution must pass a required OSCE exam at the end of their third year. As part of this exam, we developed a new 20-minute computer-based station to assess students' EBM skills. Using a specific case scenario, we asked the students to (1) ask a question using the population/intervention/comparison/outcome (PICO) framework; (2) generate appropriate search terms, given a specific question; and (3) select an appropriate abstract to answer a given question and state why two other abstracts were not appropriate. Prior to the assessment, we determined grading and passing criteria for each of the three components and for the station overall. Of the 140 students who completed the station, the percentages that passed the components were 71%, 81%, and 49% respectively, with only 29% passing all three parts. Preliminary analysis of psychometric properties of the station shows very good to excellent interrater reliability, with 65%, 67%, and 94% agreement on the scoring for the components, and kappas of.64,.82, and.94, respectively.

DISCUSSION

Although there are many curricula for teaching EBM concepts, there are few tools to measure whether students are competent in applying their EBM skills. Our pilot station appears to be an innovative and promising tool to measure several EBM skills independently. By being computer-based, it is relatively simple to administer, grade, and evaluate. While preliminary data show good inter-rater reliability with our use of a single case, future work will include further testing of reliability and assessment of different types of cases. We will also use the results of this assessment to drive continuous improvement in our EBM curriculum. The students who completed this pilot station had not received an extensive formal EBM curriculum, whereas future groups will. We also will explore whether scores on our station correlate with those on other OSCE stations that also assess critical thinking skills, or if scores correlate with a student's clinical grades or overall class standing. We hope to test these hypotheses: (1) skills used in EBM are useful and valid measures of critical thinking abilities in learners and (2) tools such as ours will help to measure these essential competencies.

Infant sleep placement after the back to sleep campaign

OBJECTIVES

The Back to Sleep campaign has been credited with recent declines in the incidence of sudden infant death syndrome. Using survey data for the 1996-1998 birth cohorts, this epidemiologic study examines infant sleep position in a large, population-based sample.

DATA AND METHODS

Data concerning infant sleep position are drawn from the 1996-1998 Pregnancy Risk Assessment Monitoring System for 15 states. Weighted multiple logistic regression analysis is used to examine correlates of infant sleep position.

RESULTS

The prevalence of prone infant sleeping significantly declined between 1996 and 1998 (adjusted odds ratio [AOR] = 0.70; 95% confidence interval [CI] = [0.63, 0.78]). African Americans were more likely than non-Hispanic whites to sleep prone, (AOR = 1.45; 95% CI = 1.33,1.59), and were less likely to sleep supine (AOR = 0.52; 95% CI = 0.48, 0.57). Hispanic/Latinos were less likely overall than non-Hispanic whites to sleep prone (AOR = 0.81; 95% CI = 0.69, 0.95), but were also less likely to sleep supine (AOR = 0.78; 95% CI = 0.69, 0.87). Adherence to sleep position recommended by the American Academy of Pediatrics increased sharply among Hispanic/Latino infants. Very low birth weight infants and infants in larger families were less likely to sleep in the recommended supine position. Infants born between 1001 and 1500 g (AOR = 0.67; 95% CI = 0.57, 0.79), and extremely low birth weight infants between 500 and 1000 g (AOR = 0.57; 95% CI = 0.45, 0.72) were especially unlikely to sleep supine. Infants in households with more than 3 other children (AOR = 1.72; 95% CI = 1.08, 2.74) were more likely to sleep prone.

CONCLUSIONS

The prevalence of supine infant sleep increased between 1996 and 1998. Low adherence to sleep position recommendations of the American Academy of Pediatrics among African Americans, very low birth weight infants, and infants in large families remain public health concerns.