A Milestone in the Milestones Movement: the JGME Milestones Supplement

Anaesthesia training designs across Europe: A survey-based study from the trainees committee of the European Society of Anaesthesiology and Intensive Care

BACKGROUND

Anaesthesiology training programs in Europe vary in duration, content, and requirements for completion. This survey-based study conducted by the Trainees Committee of the European Society of Anaesthesiology and Intensive Care explores current anaesthesia training designs across Europe.

METHODS

Between May and July 2018, we sent a 41-item online questionnaire to all National Trainee Representatives, members of the National Anaesthesiologists Societies Committee, and Council Representatives of the European Society of Anaesthesiology and Intensive Care (ESAIC) of all member countries. We cross-validated inconsistent data with different country representatives.

RESULTS

Forty-three anaesthesiologists from all 39 associated ESAIC countries completed the questionnaire. Results showed considerable variability in teaching formats, frequency of teaching sessions during training, and differences in assessments made during and at the end of training. The reported duration of training was 60 months in 59% (n = 23) of participating countries, ranging from 24 months in Russia and Ukraine to 84 months in the UK.

CONCLUSION

This study shows the significant differences in anaesthesiology training formats across Europe, and highlights the importance of developing standardised training programs to ensure a consistent level of training and to improve patient safety. This study provides valuable insights into European anaesthesia training, and underlines the need for further research and collaboration to improve requirements.

Education Research in Orthopaedic Surgery

With the ever-changing dynamics of surgical training, it is essential for the content of educational research to evolve simultaneously. This study aimed to assess the current state of scholarly work in orthopaedic training education and to identify particular educational topics that are trending in the literature.

Predicting Performance at Graduation From Early ACGME Milestone Ratings: Longitudinal Learning Analytics in Professionalism and Communication in Vascular Surgery

OBJECTIVE

Program directors in surgical disciplines need more tools from the ACGME to help them use Milestone ratings to improve trainees' performance. This is especially true in competencies that are notoriously difficult to measure, such as professionalism (PROF) and interpersonal and communication skills (ICS). It is now widely understood that skills in these two areas have direct impact on patient care outcomes. This study investigated the potential for generating early predictors of final Milestone ratings within the PROF and ICS competency categories.

DESIGN

This retrospective cohort study utilized Milestone ratings from all ACGME-accredited vascular surgery training programs, covering residents and fellows who completed training in June 2019. The outcome measure studied was the rate of achieving the recommended graduation target of Milestone Level 4 (possible range: 1-5), while the predictors were the Milestone ratings attained at earlier stages of training. Predictive probability values (PPVs) were calculated for each of the 3 PROF and two ICS sub-competencies to estimate the probability of trainees not reaching the recommended graduation target based on their previous Milestone ratings.

SETTING

All ACGME-accredited vascular surgery training programs within the United States.

PARTICIPANTS

All trainees completing a 2 year vascular surgery fellowship (VSF) in June 2019 (n = 119) or a 5 year integrated vascular surgery residency (IVSR) in June 2019 (n = 52) were included in the analyses.

RESULTS

The overall rate of failing to achieve the recommended graduation target across all PROF and ICS sub-competencies ranged from 7.7% to 21.8% of all trainees. For trainees with a Milestone rating at ≤ 2.5 with 1 year remaining in their training program, the predictive probability of not achieving the recommended graduation target ranged from 37.0% to 71.5% across sub-competencies, with the highest risks observed under PROF for "Administrative Tasks" (71.5%) and under ICS for "Communication with the Healthcare Team" (56.7%).

CONCLUSIONS

As many as 1 in 4 vascular surgery trainees did not achieve the ACGME vascular surgery Milestones targets for graduation in at least one of the PROF and ICS sub-competencies. Biannual ACGME Milestone assessment ratings of PROF and ICS during early training can be used to predict achievement of competency targets at time of graduation. Early clues to problems in PROF and ICS enable programs to address potential deficits early in training to ensure competency in these essential non-technical skills prior to entering unsupervised practice.

Comparing Entrustment Decision-Making Outcomes of the Core Entrustable Professional Activities Pilot, 2019-2020

IMPORTANCE

Gaps in readiness for indirect supervision have been identified for essential responsibilities encountered early in residency, presenting risks to patient safety. Core Entrustable Professional Activities (EPAs) for entering residency have been proposed as a framework to address these gaps and strengthen the transition from medical school to residency.

OBJECTIVE

To assess progress in developing an entrustment process in the Core EPAs framework.

DESIGN, SETTING, AND PARTICIPANTS

In this quality improvement study in the Core EPAs for Entering Residency Pilot, trained faculty made theoretical entrustment determinations and recorded the number of workplace-based assessments (WBAs) available for each determination in 2019 and 2020. Four participating schools attempted entrustment decision-making for all graduating students or a randomly selected subset of students. Deidentified, individual-level data were merged into a multischool database.

INTERVENTIONS

Schools implemented EPA-related curriculum, WBAs, and faculty development; developed systems to compile and display data; and convened groups to make theoretical summative entrustment determinations.

MAIN OUTCOMES AND MEASURES

On an EPA-specific basis, the percentage of students for whom an entrustment determination could be made, the percentage of students ready for indirect supervision, and the volume of WBAs available were recorded.

RESULTS

Four participating schools made 4525 EPA-specific readiness determinations (2296 determinations in 2019 and 2229 determinations in 2020) for 732 graduating students (349 students in 2019 and 383 students in 2020). Across all EPAs, the proportion of determinations of "ready for indirect supervision" increased from 2019 to 2020 (997 determinations [43.4%] vs 1340 determinations [60.1%]; 16.7 percentage point increase; 95% CI, 13.8-19.6 percentage points; P < .001), as did the proportion of determinations for which there were 4 or more WBAs (456 of 2295 determinations with WBA data [19.9%] vs 938 [42.1%]; 22.2 percentage point increase; 95% CI, 19.6-24.8 percentage points; P < .001). The proportion of EPA-specific data sets considered for which an entrustment determination could be made increased from 1731 determinations (75.4%) in 2019 to 2010 determinations (90.2%) in 2020 (14.8 percentage point increase; 95% CI, 12.6-16.9 percentage points; P < .001). On an EPA-specific basis, there were 5 EPAs (EPA 4 [orders], EPA 8 [handovers], EPA 10 [urgent care], EPA 11 [informed consent], and EPA 13 [patient safety]) for which few students were deemed ready for indirect supervision and for which there were few WBAs available per student in either year. For example, for EPA 13, 0 of 125 students were deemed ready in 2019 and 0 of 127 students were deemed ready in 2020, while 0 determinations in either year included 4 or more WBAs.

CONCLUSIONS AND RELEVANCE

These findings suggest that there was progress in WBA data collected, the extent to which entrustment determinations could be made, and proportions of entrustment determinations reported as ready for indirect supervision. However, important gaps remained, particularly for a subset of Core EPAs.

Medical Residency Milestones: Competencies in Informatics, Library, and Evidence-Based Practice

The Accreditation Council for Graduate Medical Education (ACGME) sets standards known as Milestones and monitors the progress of medical residents as they advance toward medical practice in their specialties. Health sciences librarians need to train medical residents in certain competency areas to help reach the Milestone standards. This project analyzed the Milestones related to informatics, library, and evidence-based practice (EBP) skills to identify core and optional library-related curricular elements that can be integrated into different medical specialty residencies. The authors collected key competency documents from ACGME and from those specialties representing 2% or more of the residencies in the United States. Then, they compared and contrasted those Milestones related to informatics, library, and EBP competency skills. Most relevant Milestones were categorized under the fifth broad ACGME competency area of "Practice Based Learning and Improvement." The Milestones followed developmental patterns, reflecting residents' increased sophistication in meeting these competencies as they advanced in their specialties. The curriculum was designed to meet the residents' learning needs at each progressive Milestone.

Milestone Approach to Designing a Point-of-Care Ultrasound Curriculum for Transition-to-Residency Programs in the United States

Phenomenon: Point-of-care ultrasound is fast becoming standard clinical bedside practice for diverse specialties. Medical schools are responding by adding ultrasound education, though the majority use it to supplement the learning of basic sciences. Point-of-care ultrasound practice-based clinical skills education is rare. There also is a lack of standardization across curricula, leading to much variability in the ultrasound skills that medical students from different schools bring to residency. To best inform a point-of-care ultrasound curriculum for our Transition-to Residency program, we investigated literature on 1) how medical students are being prepared for use of point-of-care ultrasound in clinical practice, 2) what skills are being taught, 3) what point-of-care ultrasound skills residency programs expect from incoming residents. Approach: We reviewed literature to identify curricula in U.S. medical schools that teach the concepts, knowledge, and skills related to point-of-care ultrasound. We also mapped point-of-care ultrasound expectations set forth by the Entrustable Professional Activities for undergraduate medical education to the specialty-specific milestones identified by the Accreditation Council for Graduate Medical Education. Additionally, we reviewed specialty-specific professional organizations for position statements and guidelines describing the point-of-care ultrasound skills expected for practicing physicians in their respective specialties. The goal was to identify any needs and gaps in education regarding point-of-care ultrasound across the undergraduate to graduate medical education continuum to practice. Findings: We found seven published point-of-care ultrasound curricula for medical students. There was wide variability in these curricula regarding what point-of-care ultrasound content is being taught, as well as when and how this skill is taught. No Entrustable Professional Activity listed point-of-care ultrasound as a skill requirement for graduating medical students. For graduate medical education, there was wide variability across specialties in residency milestones related to point-of-care ultrasound; some (e.g., emergency medicine) listed extensive milestones while others (e.g., internal medicine) listed none. However, we found that many specialty-specific professional organizations do list detailed point-of-care ultrasound expectations for their practicing physicians. Insights: As point-of-care ultrasound is fast becoming common practice across many specialties, standardization of education and related competencies-similar to other clinical skills training-is necessary across medical schools. Mapping point-of-care ultrasound expectations to current teaching across the continuum from undergraduate to graduate medical education may allow schools to tailor point-of-care ultrasound training for Transition-to-Residency programs. We provide a sample pilot point-of-care ultrasound curriculum that we designed for our Transition-to-Residency course.

Development of Competency in Needle-Guided Procedures Through the Use of Soft-Embalmed Cadavers

This study evaluated soft-embalmed human cadavers for ultrasound needle-guidance training. Second-year medical students used peripherally inserted central catheter (PICC) line and central line models and a Thiel-embalmed cadaver for training and provided feedback. Most (85%) agreed the cadaver was useful. There were positive correlations between performing the PICC line (r=0.44, P=.11) and central line (r=0.63, P=.03) procedures on a cadaver and in a clinical setting. Thiel-embalmed cadavers may provide important hands-on training.

Emergency Medicine/Critical Care Medicine (EM/CCM) Trainees' Performance on Standardized Critical Care Medicine Examinations: A Ten-Year Review

BACKGROUND

Emergency Medicine/Critical Care Medicine (EM/CCM) trainees may obtain board certification through Internal Medicine (American Board of Internal Medicine [ABIM]), Surgery (American Board of Surgery [ABS]), and Anesthesiology (American Board of Anesthesiology [ABA]). However, EM/CCM trainees experience challenges, including: 1) additional training requirements and 2) an unwillingness to accept EM graduates by many programs.

OBJECTIVES

We sought to: 1) compare EM/CCM knowledge acquisition to medicine (Internal Medicine [IM]/CCM), surgery (surgical critical care [SCC]), and anesthesiology (anesthesiology critical care medicine [ACCM]) Fellows at the local and national level using the Multidisciplinary Critical Care Knowledge Assessment Program (MCCKAP) in-service examination as an objective measure; and 2) compare American Board of Medical Specialties (ABMS) pass rates for EM/CCM.

METHODS

Single-center retrospective analysis comparing scores obtained by EM/CCM on the MCCKAP examination with SCC and ACCM over a 10-year period. Scores are presented as means with standard deviations. We performed similar analysis on ABMS examination pass rates.

RESULTS

There were 117 MCCKAP scores (37 EM/CCM; 80 SCC and ACCM) evaluated. EM/CCM mean score 562.4 (SD 67.4); SCC and ACCM mean score 505.3, (SD 87.5) at the institutional level (p < 0.001). Similarly, EM/CCM scored higher than the national mean (562.4, SD 67.4 vs. 500 SD 100, p < 0.001). Nationally, ABIM-CCM board certification rate was 91.2% for 137 EM/CCM, compared with 93.2% for IM/CCM (p = 0.22); 28 EM/CCM have obtained ABA-CCM board certification with rates similar to ACCM (90.4 vs. 89.3%; p = 0.85).

CONCLUSIONS

EM/CCM Fellows demonstrate successful knowledge acquisition both locally and at a national level. EM/CCM achieve ABMS pass rates similar to other CCM trainees. The current arbitrary additional training requirements placed on EM/CCM should be removed.

Peer Coaching as a Faculty Development Tool: A Mixed Methods Evaluation

BACKGROUND

In the era of competency-based assessment, medical education faculty are frequently challenged to develop unique teaching approaches. One method to address faculty development needs in a real-time clinical learning environment is peer coaching.

OBJECTIVE

We implemented and evaluated a faculty development program involving peer observation and feedback for attending physicians.

METHODS

Hospital internal medicine faculty assigned to a teaching service were recruited for the study. Participants voluntarily agreed to observe and be observed by a peer attending physician during a 2-week block of teaching rounds. When serving in the coaching role, faculty were asked to observe 4 separate occasions using an observation tool based on the Stanford Faculty Development Program framework to guide feedback. An outside consultant facilitated a focus group and completed a qualitative content analysis to categorize all participants' experiences during the faculty development activity.

RESULTS

Of the 22 eligible faculty, 14 (64%) agreed to participate by committing to 6 to 8 hours observing another faculty member during rounds, 2 feedback sessions, and 90 minutes to provide program feedback during a focus group. The analysis of the focus group revealed favorable reactions to the faculty development program, including (1) observed attending awareness of unrecognized habits; (2) personalized teaching tips for the observed attending to improve teaching quality based on individual style/preferences; and (3) exposure to new teaching techniques.

CONCLUSIONS

An inpatient-based peer-coaching faculty development program was acceptable and feasible for a majority of faculty and may improve individual teaching effectiveness among conventionally trained physicians.

Using an Entrustable Professional Activity to Assess Consultation Requests Called on an Internal Medicine Teaching Service

INTRODUCTION

The Accreditation Council for Graduate Medical Education's milestones require internal medicine residents to have competency in calling consults. Based on a literature review, we developed an Entrustable Professional Activity (EPA) to delineate the knowledge, skills, and attitudes required for a consultation request and, building on the EPA, implemented an assessment instrument to provide feedback to interns calling consultation requests and assess the quality of their consult questions and the level of supervision required in performing this milestone.

METHODS

Assessments were done on internal medicine inpatient teaching services. Consultation requests were performed by interns and observed by residents using the assessment instrument. Feedback was provided to the interns. Interns then completed a self-reflection instrument based on the feedback.

RESULTS

Twenty-six paired observations were collected over three 1-month rotations. There was a moderate positive correlation (r = .43) comparing resident and intern responses to how they felt about the intern's ability to make a consultation request. There was a strong positive correlation (r = .65) comparing resident opinion of how strong the intern's ability in calling a consult to how well the consult question used the PICO (patient, intervention, comparators, outcomes of interest) framework. Twenty-five out of 28 interns (89%) said they would make a change during their next consultation request due to feedback from their resident.

DISCUSSION

Our EPA-based assessment instrument provided an opportunity to give interns feedback and to assess the quality of the consultation requests they made.

A National Study of Longitudinal Consistency in ACGME Milestone Ratings by Clinical Competency Committees: Exploring an Aspect of Validity in the Assessment of Residents' Competence

PURPOSE

To investigate whether clinical competency committees (CCCs) were consistent in applying milestone ratings for first-year residents over time or whether ratings increased or decreased.

METHOD

Beginning in December 2013, the Accreditation Council for Graduate Medical Education (ACGME) initiated a phased-in requirement for reporting milestones; emergency medicine (EM), diagnostic radiology (DR), and urology (UR) were among the earliest reporting specialties. The authors analyzed CCC milestone ratings of first-year residents from 2013 to 2016 from all ACGME-accredited EM, DR, and UR programs for which they had data. The number of first-year residents in these programs ranged from 2,838 to 2,928 over this time period. The program-level average milestone rating for each subcompetency was regressed onto the time of observation using a random coefficient multilevel regression model.

RESULTS

National average program-level milestone ratings of first-year residents decreased significantly over the observed time period for 32 of the 56 subcompetencies examined. None of the other subcompetencies showed a significant change. National average in-training examination scores for each of the specialties remained essentially unchanged over the time period, suggesting that differences between the cohorts were not likely an explanatory factor.

CONCLUSIONS

The findings indicate that CCCs tend to become more stringent or maintain consistency in their ratings of beginning residents over time. One explanation for these results is that CCCs may become increasingly comfortable in assigning lower ratings when appropriate. This finding is consistent with an increase in confidence with the milestone rating process and the quality of feedback it provides.

Implementing Competency-Based Medical Education in Internal Medicine Residency Training Program: the Process and Impact on Residents' Satisfaction

BACKGROUND

Graduate medical education is shifting from the traditional apprenticeship model to a competency-driven model. Here we describe the design and implementation of competency-based medical education (CBME) in an internal medicine residency program, and report satisfaction survey results.

METHODS

We redesigned the residency curriculum as CBME to be resident-centred, systematic, focused on general internal medicine, to provide experience in various care setting, and work-based assessment. In the second year of this CBME transition, we surveyed residents' overall satisfaction using 5-point Likert scale. Feedback on their training program was also analysed.

RESULTS

The overall satisfaction score was 3.24 and thirteen residents (61.9%) answered that the preceptor's practical training in an educational atmosphere and improvement through training were the merits of the training program. However, residents complained about the working condition such as work overload.

CONCLUSION

With the CBME implementation, most residents expressed satisfaction with the hospital's educational environment but they suffered from overwork. Further efforts to improve the educational program and environment are warranted.

The Clinical Competency Committee in Adult Cardiothoracic Anesthesiology-Perspectives From Program Directors Around the United States

The clinical competency committee offers a fellowship program a structured approach to assess the clinical performance of each trainee in a comprehensive fashion This special article examines the structure and function of this important committee in detail. Furthermore, the strategies for the optimal functioning of this committee are also discussed as a way to enhance the overall quality of the fellowship program.

Predictability of Clinical Knowledge Through Mobile App-based Simulation for the Treatment of Pediatric Septic Arthritis: A Pilot Study

BACKGROUND

Recently the American Board of Orthopaedic Surgery and the Accreditation Council of Graduate Medical Education have identified the treatment of septic arthritis of the hip in children as a milestone skill for all US orthopaedic residents. The purpose of this study was to test correlation between clinical knowledge and examination score on a mobile app-based training module for the treatment of pediatric septic hip arthritis.

METHODS

A 4-part simulation model on surgical decision-making associated with the treatment of pediatric septic arthritis was developed through expert consensus. Orthopaedic trainees participating in the "Top Gun" program of the 2015 and 2016 International Pediatric Orthopaedic Symposiums were recruited to participate in this pilot study. Trainees completed a presimulation quiz on their knowledge of diagnosis, arthrocentesis, and surgical irrigation and debridement on a pediatric patient presenting with septic arthritis of the hip. Trainees then completed the 4-part simulation on the mobile app. Pearson correlation analysis was used to assess the relationship between the quiz and the simulation.

RESULTS

A total of 53 orthopaedic residents and fellows participated in the simulation. Median quiz score was 87 points [interquartile range (IQR), 81 to 94] before the intervention and 100 points (IQR, 94 to 100) postintervention. The median simulation test score was 89 (IQR, 81 to 92) which demonstrated a positive correlation with the postintervention quiz (r=0.44, P<0.001). The preintervention metrics demonstrated a positive correlation with postintervention metrics (r=0.53, P<0.001).

CONCLUSIONS

This study revealed a statistically significant positive correlation between the mobile app simulation and the clinical knowledge of the participants, as well as the ability to improve knowledge about a procedure during the testing period. These findings support the ability for the mobile app to test clinical knowledge. In the current environment of decreased work hours and patient exposure for orthopaedic trainees, mobile app-based simulation has the potential to safely aid in assessment of orthopaedic residents and fellows.

Using an alumni survey to target improvements in an emergency medicine training programme

INTRODUCTION

The Accreditation Council for Graduate Medical Education (ACGME) is the governing body responsible for accrediting graduate medical training programme in the USA. The Emergency Medicine Milestones (EM-Milestones) were developed by the ACGME and American Board of Emergency Medicine as a guide and monitoring tool for the knowledge, skills, abilities and experiences to be acquired during training. Alumni surveys have been reported as a valuable resource for training programme to identify areas for improvement; however, there are few studies regarding programme improvement in emergency medicine. We aimed to use the EM-Milestones, adapted as an alumni self-assessment survey, to identify areas for training programme improvement.

METHODS

This study was conducted at an urban, academic affiliated, community hospital in New York city with an emergency medicine training programme consisting of 30 residents over 3 years. Alumni of our emergency medicine training programme were sent an EM-Milestones-based self-assessment survey. Participants evaluated their ability in each EM-Milestones subcompetency on a Likert scale. Data were analysed using descriptive statistics.

RESULTS

Response rate was 74% (69/93). Alumni reported achieving the target performance in 5/6 general competencies, with Systems-Based Practice falling below the target performance. The survey further identified 6/23 subcompetencies (Pharmacotherapy, Ultrasound, Wound Management, Patient Safety, Systems-Based Management and Technology) falling below the target performance level.

DISCUSSION

Alumni self-evaluation of competence using the EM-Milestones provides valuable information concerning confidence to practice independently; these data, coupled with regular milestone evaluation of existing trainees, can identify problem areas and provide a blueprint for targeted programme improvement.

From Communication Skills to Skillful Communication: A Longitudinal Integrated Curriculum for Critical Care Medicine Fellows

PROBLEM

Communication with patients and families in critical care medicine (CCM) can be complex and challenging. A longitudinal curricular model integrating multiple techniques within classroom and clinical milieus may facilitate skillful communication across diverse settings.

APPROACH

In 2014-2015, the authors developed and implemented a curriculum for CCM fellows at the Cumming School of Medicine, University of Calgary, to promote the longitudinal development of skillful communication. A departmental needs assessment informed curriculum development. Five 4-hour classroom sessions were developed: basic communication principles, family meetings about goals and transitions of care, discussing patient safety incidents, addressing conflict, and offering organ donation. Teaching methods-including instructor-led presentations incorporating a consistent framework for approaching challenging conversations, simulation and clinical practice, and feedback from peers, trained facilitators, family members, and clinicians-supported integration of skills into the clinical setting and longitudinal development of skillful communication. Seven fellows participated during the first year of the curriculum.

OUTCOMES

CCM fellows engaged enthusiastically in the program, commented that the framework provided was helpful, and highly valued the opportunity to practice challenging communication scenarios, learn from observing their peers, and receive immediate feedback.

NEXT STEPS

More detailed accounts of fellows', patients', and family members' experiences will be obtained to guide curricular development. The curriculum will be expanded to involve other members of the multidisciplinary intensive care unit team, and faculty education initiatives will be offered to enhance the quality of the feedback provided. The impact of the curriculum on initial skill development, retention, and progression will be assessed.

Goals of care conversation teaching in residency - a cross-sectional survey of postgraduate program directors

BACKGROUND

Residents are commonly involved in establishing goals of care for hospitalized patients. While education can improve the quality of these conversations, whether and how postgraduate training programs integrate such teaching into their curricula is not well established. The objective of this study was to characterize perceptions of current teaching and assessment of goals of care conversations, and program director interest in associated curricular integration.

METHODS

An electronic survey was sent to all postgraduate program directors at the University of Calgary. Quantitative data was analyzed using descriptive statistics and qualitative comments were analyzed using thematic analysis.

RESULTS

The survey response rate was 34% (22/64). Formal goals of care conversation teaching is incorporated into 63% of responding programs, and most commonly involves lectures. Informal teaching occurs in 86% of programs, involving discussion, direct observation and role modeling in the clinical setting. Seventy-three percent of programs assess goals of care conversation skills, mostly in the clinical setting through feedback. Program directors believe that over two-thirds of clinical faculty are prepared to teach goals of care conversations, and are interested in resources to teach and assess goals of care conversations. Themes that emerged include 1) general perceptions, 2) need for teaching, 3) ideas for teaching, and 4) assessment of goals of care conversations.

CONCLUSIONS

The majority of residency training programs at the University of Calgary incorporate some goals of care conversation teaching and assessment into their curricula. Program directors are interested in resources to improve teaching and assessment of goals of care conversations.

Mapping Direct Observations From Objective Structured Clinical Examinations to the Milestones Across Specialties

BACKGROUND

Little is known about residents' performance on the milestones at the institutional level. Our institution formed a work group to explore this using an institutional-level curriculum and residents' evaluation of the milestones.

OBJECTIVE

We assessed whether beginner-level milestones for interpersonal and communication skills (ICS) related to observable behaviors in ICS-focused objective structured clinical examinations (OSCEs) for postgraduate year (PGY) 1 residents across specialties.

METHODS

The work group compared ICS subcompetencies across 12 programs to identify common beginner-level physician-patient communication milestones. The selected ICS milestone sets were compared for common language with the ICS-OSCE assessment tool-the Kalamazoo Essential Elements of Communication Checklist-Adapted (KEECC-A). To assess whether OSCE scores related to ICS milestone scores, all PGY-1 residents from programs that were part of Next Accreditation System Phase 1 were identified; their OSCE scores from July 2013 to June 2014 and ICS subcompetency scores from December 2014 were compared.

RESULTS

The milestones for 10 specialties and the transitional year had at least 1 ICS subcompetency that related to physician-patient communication. The language of the ICS beginner-level milestones appears similar to behaviors outlined in the KEECC-A. All 60 residents with complete data received at least a beginner-level ICS subcompetency score and at least a satisfactory score on all 3 OSCEs.

CONCLUSIONS

The ICS-OSCE scores for PGY-1 residents appear to relate to beginner-level milestones for physician-patient communication across multiple specialties.

Reflections in a time of transition: orthopaedic faculty and resident understanding of accreditation schemes and opinions on surgical skills feedback

INTRODUCTION

Orthopaedic surgery is one of the first seven specialties that began collecting Milestone data as part of the Accreditation Council for Graduate Medical Education's Next Accreditation System (NAS) rollout. This transition from process-based advancement to outcome-based education is an opportunity to assess resident and faculty understanding of changing paradigms, and opinions about technical skill evaluation.

METHODS

In a large academic orthopaedic surgery residency program, residents and faculty were anonymously surveyed. A total of 31/32 (97%) residents and 29/53 (55%) faculty responded to Likert scale assessments and provided open-ended responses. An internal end-of-rotation audit was conducted to assess timeliness of evaluations. A mixed-method analysis was utilized, with nonparametric statistical testing and a constant-comparative qualitative method.

RESULTS

There was greater familiarity with the six core competencies than with Milestones or the NAS (p<0.05). A majority of faculty and residents felt that end-of-rotation evaluations were not adequate for surgical skills feedback. Fifty-eight per cent of residents reported that end-of-rotation evaluations were rarely or never filled out in a timely fashion. An internal audit demonstrated that more than 30% of evaluations were completed over a month after rotation end. Qualitative analysis included themes of resident desire for more face-to-face feedback on technical skills after operative cases, and several barriers to more frequent feedback.

DISCUSSION

The NAS and outcome-based education have arrived. Residents and faculty need to be educated on this changing paradigm. This transition period is also a window of opportunity to address methods of evaluation and feedback. In our orthopaedic residency, trainees were significantly less satisfied than faculty with the amount of technical and surgical skills feedback being provided to trainees. The quantitative and qualitative analyses converge on one theme: a desire for frequent, explicit, timely feedback after operative cases. To overcome the time-limited clinical environment, feedback tools need to be easily integrated and efficient. Creative solutions may be needed to truly achieve outcome-based graduate medical education.

Medical Student Core Clinical Ultrasound Milestones: A Consensus Among Directors in the United States

OBJECTIVES

Many medical schools are implementing point-of-care ultrasound in their curricula to help augment teaching of the physical examination, anatomy, and ultimately clinical management. However, point-of-care ultrasound milestones for medical students remain unknown. The purpose of this study was to formulate a consensus on core medical student clinical point-of-care ultrasound milestones across allopathic and osteopathic medical schools in the United States. Directors who are leading the integration of ultrasound in medical education (USMED) at their respective institutions were surveyed.

METHODS

An initial list of 205 potential clinical ultrasound milestones was developed through a literature review. An expert panel consisting of 34 USMED directors across the United States was used to produce consensus on clinical ultrasound milestones through 2 rounds of a modified Delphi technique, an established anonymous process to obtain consensus through multiple rounds of quantitative questionnaires.

RESULTS

There was a 100% response rate from the 34 USMED directors in both rounds 1 and 2 of the modified Delphi protocol. After the first round, 2 milestones were revised to improve clarity, and 9 were added on the basis of comments from the USMED directors, resulting in 214 milestones forwarded to round 2. After the second round, only 90 milestones were found to have a high level of agreement and were included in the final medical student core clinical ultrasound milestones.

CONCLUSIONS

This study established 90 core clinical milestones that all graduating medical students should obtain before graduation, based on consensus from 34 USMED directors. These core milestones can serve as a guide for curriculum deans who are initiating ultrasound curricula at their institutions. The exact method of implementation and competency assessment needs further investigation.

Genetic Counseling Milestones: A Framework for Student Competency Evaluation

Graduate medical education has recently increased focus on the development of medical specialty competency milestones to provide a targeted tool for medical resident evaluation. Milestones provide developmental assessment of the attainment of competencies over the course of an educational program. An educational framework is described to explore the development of Genetic Counseling Milestones for the evaluation of the development of genetic counseling competencies by genetic counseling students. The development of Genetic Counseling Milestones may provide a valuable tool to assess genetic counseling students across all program activities. Historical educational context, current practices, and potential benefits and challenges in the development of Genetic Counseling Milestones are discussed.

Milestones for Internal Medicine Sub-interns

BACKGROUND

As residency programs move toward measuring milestones for competency-based education assessment, medical schools will need to collaborate with residencies to determine competencies for graduating students. The objective of this study is to define the educational milestones for fourth-year medical students during an Internal Medicine sub-internship.

METHODS

A cross-sectional Internet-based survey (with attention to validity evidence) was developed in early 2013 and administered to Internal Medicine attendings and Internal Medicine sub-interns working on an inpatient team at 3 academic medical centers. With the purpose to determine the milestones for sub-interns, items asked respondents what responsibilities a sub-intern could be entrusted to perform without direct supervision.

RESULTS

Faculty responded that behaviors sub-interns could perform with indirect supervision were mostly at the "reporter" level, including completing a history and physical examination and collecting data such as test results. Other skills such as venipuncture and some communication skills such as calling consults, providing patient counseling, responding to pages, and creating discharge instructions were examples of tasks in which the majority of faculty felt that students were progressing toward unsupervised practice. Behaviors where the majority of faculty would always supervise a medical student performance included performance on the "interpreter" level, including interpreting electrocardiograms, significant physical examination findings, and laboratory results. Medical students less commonly noted needing supervision on the majority of the items when compared with faculty.

CONCLUSION

Tasks in the reporter domain such as taking a history, collecting medical records, and reporting results can be characterized as medical student milestones.

Medical student communication skills and specialty choice

OBJECTIVE

The aim of this study was to determine if communication skills differ for medical students entering person or technique-oriented specialties.

METHODS

Communication ratings by clerkship preceptors on an institutionally required end of clerkship medical student performance evaluation (SPE) form were compiled for 2011/2012 academic year (Class of 2013). M3 clerkships and the Class of 2013 match appointments were categorized as person or technique-oriented clerkships/specialties. Mean differences in SPE communication scores were determined by analyses of variance (ANOVA) and independent t tests. Score associations were determined by Pearson correlations. Inter-item reliability was reported with Cronbach alpha.

RESULTS

The Class of 2013 match appointments were as follows: person-oriented (N = 91) and technique-oriented (N = 91) residency specialties. There was no significant difference in mean communication scores for medical students who entered person-oriented (mean 7.8, SD 0.4) versus technique-oriented (mean 7.9, SD 0.4) specialties (p = 0.258) or for person-oriented clerkship (mean 7.8, SD 0.4) versus technique-oriented clerkship (mean 7.9, SD 0.6) ratings for medical students who matched into person-oriented specialties (p = 0.124). Medical students who matched into technique-oriented specialties (mean 8.1, SD 0.5) received significantly higher (p = 0.001) communication ratings as compared with those matching into person-oriented specialties (mean 7.8, SD 0.5) from technique-oriented clerkships.

CONCLUSIONS

Communication with patients and families is a complex constellation of specific abilities that appear to be influenced by the rater's specialty. Further study is needed to determine if technique-oriented specialties communication skill rating criteria differ from those used by raters from person-oriented specialties.

The Courteous Consult: A CONSULT Card and Training to Improve Resident Consults

BACKGROUND

Communication and courtesy are important elements of consultations, but there is limited published data about the quality of trainee consults.

OBJECTIVES

This study assessed residents' views on consult interactions, evaluated the impact of the consult interactions on patient care, and developed and implemented a pocket card and training on trainee consults.

METHODS

We surveyed resident and fellow physicians at Mount Sinai Hospital to assess perceptions, created a CONSULT card that uses a mnemonic for key elements, and developed a training session for how to call consults. We also conducted a consult training session using the CONSULT card as part of orientation in 2011 for all interns. We assessed the acceptability, feasibility, and sustainability of this intervention.

RESULTS

Of 1001 trainees, 403 (40%) responded. Respondents reported that the most important components of calling consults included giving patient name, medical record number, and location (91%), and giving a clear question/reason (89%). Respondents also reported that these behaviors are done consistently for only 64%, and 10% of consults, respectively. Trainees reported that consult interactions affect the timeliness of treatment (62%), timeliness of tests performed (57%), appropriateness of diagnosis (56%), and discharge planning (49%). Approximately 300 interns attended the consult training session, and their feedback demonstrated acceptability and utility of the session.

CONCLUSIONS

Trainees believe that consult interactions impact patient care, but important components of the consult call are often missing. Our training and CONSULT card is an acceptable, feasible, and novel training intervention. Once developed, the training session and CONSULT card require minimal faculty time to deliver.

Medical Board Complaints against Physicians Due to Communication: Analysis of North Carolina Medical Board Data, 2002–2012

Anecdotal evidence suggests that communication issues are one of the primary reasons for physician complaints, but quantitative studies have yet to examine this assertion. The North Carolina Medical Board's Complaint Department maintains data on physician complaints and categorizes each complaint based on its primary cause. Using data from 2002–2012, our research focused on complaints against physicians licensed by the North Carolina board to determine the extent to which communication issues contribute to complaints against physicians. An analysis of this data reveals that physician complaints based on communication issues are consistently the most prevalent reason for complaints against physicians in the state of North Carolina. In addition, communication-based complaints account for more than one in five complaints made against North Carolina physicians. These results are discussed in light of their implications for the field of medicine as it seeks to improve patient care.

Expanding the role of objectively structured clinical examinations in nephrology training

Objectively structured clinical examinations (OSCEs) are widely used in medical education, but we know of none described that are specifically for nephrology fellowship training. OSCEs use simulation to educate and evaluate. We describe a technically simple, multidisciplinary, low-cost OSCE developed by our program that contains both examination and training features and focuses on management and clinical knowledge of rare hemodialysis emergencies. The emergencies tested are venous air embolism, blood leak, dialysis membrane reaction, and hemolysis. Fifteen fellows have participated in the OSCE as examinees and/or preceptors since June 2010. All have passed the exercise. Thirteen responded to an anonymous survey in July 2013 that inquired about their confidence in managing each of the 4 tested emergencies pre- and post-OSCE. Fellows were significantly more confident in their ability to respond to the emergencies after the OSCE. Those who subsequently saw such an emergency reported that the OSCE experience was somewhat or very helpful in managing the event. The OSCE tested and trained fellows in the recognition and management of rare hemodialysis emergencies. OSCEs and simulation generally deserve greater use in nephrology subspecialty training; however, collaboration between training programs would be necessary to validate such exercises.