Considerations for using race and ethnicity as quantitative variables in medical education research

Throughout history, race and ethnicity have been used as key descriptors to categorize and label individuals. The use of these concepts as variables can impact resources, policy, and perceptions in medical education. Despite the pervasive use of race and ethnicity as quantitative variables, it is unclear whether researchers use them in their proper context. In this Eye Opener, we present the following seven considerations with corresponding recommendations, for using race and ethnicity as variables in medical education research: 1) Ensure race and ethnicity variables are used to address questions directly related to these concepts. 2) Use race and ethnicity to represent social experiences, not biological facts, to explain the phenomenon under study. 3) Allow study participants to define their preferred racial and ethnic identity. 4) Collect complete and accurate race and ethnicity data that maximizes data richness and minimizes opportunities for researchers' assumptions about participants' identity. 5) Follow evidence-based practices to describe and collapse individual-level race and ethnicity data into broader categories. 6) Align statistical analyses with the study's conceptualization and operationalization of race and ethnicity. 7) Provide thorough interpretation of results beyond simple reporting of statistical significance. By following these recommendations, medical education researchers can avoid major pitfalls associated with the use of race and ethnicity and make informed decisions around some of the most challenging race and ethnicity topics in medical education.

Academic coaching: Insights from the medical student's perspective

Purpose: With the growing recognition of the role of coaching in competency-based medical education, many medical education training programs are investing significant resources into developing coaching programs. However, there is a lack of rigorous research on academic coaching programs in medical education and recommended coaching practices are based on expert opinion without incorporating the student perspective. The aim of this study was to investigate factors that affect a students' perception of a successful coaching experience.Materials and method: This was a qualitative study performed in November 2018 at a medical school in the United States with a formal coaching program. Appreciative inquiry was the theoretical framework used to develop the question guide. The authors facilitated two focus groups and then used Colaizzi's phenomenological analysis approach to analyze the transcripts.Results: A total of 21 students participated in the focus groups. The analysis revealed four themes that describe students' perceptions of successful coaching experiences: coach attributes, relational skills, coaching skills, and utilization of coaching. Each of these themes had specific dimensions.Conclusions: The findings from this study illuminate the student stakeholder perspectives on successful coaching relationships, and should be considered when developing a coaching program and faculty development.

Limited by our limitations

Study limitations represent weaknesses within a research design that may influence outcomes and conclusions of the research. Researchers have an obligation to the academic community to present complete and honest limitations of a presented study. Too often, authors use generic descriptions to describe study limitations. Including redundant or irrelevant limitations is an ineffective use of the already limited word count. A meaningful presentation of study limitations should describe the potential limitation, explain the implication of the limitation, provide possible alternative approaches, and describe steps taken to mitigate the limitation. This includes placing research findings within their proper context to ensure readers do not overemphasize or minimize findings. A more complete presentation will enrich the readers' understanding of the study's limitations and support future investigation.

Developing Professionalism and Professional Identity Through Unproctored, Flexible Testing

The medical education community has devoted a great deal of attention to the development of professionalism in trainees within the context of clinical training-particularly regarding trainees' handling of ethical dilemmas related to clinical care. The community, however, knows comparatively less about the development of professional behavior in medical students during the preclerkship years. In medical schools with flexible testing, students take quizzes or examinations in an unproctored setting at a time of their choosing-as long as it falls within a specified window of time. Unproctored, flexible testing offers students early opportunities to develop appropriate professional behavior. In this Perspective, the authors outline different flexible testing models from three institutions-University of Virginia School of Medicine, University of Michigan Medical School, and Icahn School of Medicine at Mount Sinai-all of which offer various levels of testing flexibility in relation to time and location. The authors' experiences with these models suggest that preclinical medical students' early development of professional behavior requires scaffolding by faculty and staff. Scaffolding involves setting clear, specific expectations for students (often through the form of an honor code), as well as active engagement and discussion with learners about the expectations and procedures for self-regulation in the academic environment.

Using a Six-Domain Framework to Include Biopsychosocial Information in the Standard Medical History

PROBLEM

The traditional approach to physicians' history taking is designed to facilitate diagnosis and treatment of biomedical conditions. However, in the 21st century, health is critically influenced by the interaction of biomedical conditions and nonbiomedical factors such as patient's ability to manage chronic disease and the social determinants of health. Interventions to expand routine history taking to include nonbiomedical factors have not been widely adopted, possibly due to the difficultly of incorporating long checklists into routine care and the inability to achieve consensus on the relevant behavioral or social determinants of health content applicable to all patients.

INTERVENTION

In 2015-2016, we introduced medical students to a 6-domain (biomedical and psychiatric conditions, behavioral health, living environment/resources, social support, and functional status) approach to history taking and instructed them to elicit information from each domain alongside the traditional approach. Students were required to obtain information from each domain in one admitting history or one daily progress note, discuss their findings with the attending physician, and involve members of the medical team in addressing concerns and barriers in the care of that patient. Students' history notes were reviewed for completeness and compared to those from a student control group. Students also completed a 10-question evaluation of the model.

CONTEXT

The intervention was conducted during a 1-month rotation on a hospitalist general medicine service from May 2015 through August 2016.

OUTCOME

Patient history and daily progress notes were collected from 38 fourth-year intervention students and compared to 24 control students on the same service from the previous year. Compared to control students, intervention students provided more patient information (p ≤ .001) in all nonbiomedical domains except behavioral health. Intervention students reported that the 6-domain model helped them identify clinical information that could be addressed with existing resources and prompted involvement of social workers, pharmacists, and nurses in care planning. They also indicated the framework added clinically valuable information and enhanced team-based care.

LESSONS LEARNED

A domain-based framework can be used by medical students to identify clinically relevant behavioral conditions and social determinants of health tailored to individual patients while avoiding long standardized checklists. Arguably, routine collection of behavioral and social determinants of health is a necessary first step in enhancing physicians' awareness and skills in working with health care teams to address nonbiomedical determinants of patients' health.

Assessing Residents' Veteran-Centered Care Skills in the Clinical Setting

BACKGROUND

Despite their placement in Veterans Health Administration centers nationwide, residents' training and assessment in veteran-centered care is variable and often insufficient.

OBJECTIVE

We assessed residents' ability to recognize and address mental health issues that affect US military veterans.

METHODS

Two unannounced standardized patient (SP) cases were used to assess internal medicine residents' veteran-centered care skills from September 2014 to March 2016. Residents were assessed on 7 domains: military history taking, communication skills, assessment skills, mental health screening, triage, and professionalism, using a 36-item checklist. After each encounter, residents completed a questionnaire to assess their ability to recognize knowledge deficits. Residents' mean scores were compared across training levels, between the 2 cases, and by SP gender. We conducted analysis of variance (ANOVA) tests to analyze mean performance differences across training levels and descriptive statistics to analyze self-assessment questionnaire results.

RESULTS

Ninety-eight residents from 2 internal medicine programs completed the encounter and 53 completed the self-assessment questionnaire. Residents performed best on professionalism (0.92 ± 0.20, percentage of the maximal score) and triage (0.87 ± 0.17), and they scored lowest on posttraumatic stress disorder (0.52 ± 0.30) and military sexual trauma (0.33 ± 0.39). Few residents reported that they sought out training to enhance their knowledge and skills in the provision of services and support to military and veteran groups beyond their core curriculum.

CONCLUSIONS

This study suggests that additional education and assessment in veteran-centered care may be needed, particularly in the areas of posttraumatic stress disorder and military sexual trauma.

Using a Modified A3 Lean Framework to Identify Ways to Increase Students' Reporting of Mistreatment Behaviors

PROBLEM

The proportion of students who experience mistreatment is significantly higher than the proportion of students who report mistreatment. Identifying ways to improve students' reporting of these incidents is one strategy for increasing opportunities to achieve resolution and prevent future occurrences.

APPROACH

The authors applied a modified A3 Lean framework to examine medical student reporting of mistreatment behaviors at the University of Michigan Medical School (UMMS) in 2013-2016. The A3 Lean framework is a stepwise approach that involves outlining the background to establish the context of the problem, describing the current condition, identifying the goal or desired outcome, analyzing causes of the problem, providing proposed countermeasures for improvement, and creating follow-up plans. The authors identified three reasons for the difference between students' experiences and reporting of mistreatment and developed five countermeasures/action plan items to address this difference.

OUTCOMES

The proportion of students reporting mistreatment at UMMS increased 21.4% between 2013 and 2016. Compared with 2013, more students in 2016 indicated not reporting because the incident did not seem important enough or because they resolved the issue on their own.

NEXT STEPS

The authors have enlisted the support of the health system's human resources department and presented the inaugural grand rounds on improving the learning environment in 2016. Among other things, they are also partnering with this team to add questions about student mistreatment and civility to the annual employee engagement survey distributed to all 20,000 employees.

Veteran-centred content in medical education

BACKGROUND

Veterans have unique experiences that warrant special consideration in health care. Unfortunately, training in veteran-centred care has not been a clear focus of medical education, and only a very small proportion of medical schools include military cultural competency in their curricula.

METHODS

We conducted an 80-minute focus group with six US veterans. Open-ended questions were used to elicit their perceptions of the health care that they receive, and how it can be improved. The audio-recording was transcribed verbatim and coded for thematic content. A phenomenological analytic approach was used to analyse the 31-page transcript and arrive at the final themes.

RESULTS

Former service members from various periods of conflict (e.g. World War II, Vietnam, Persian Gulf) offered key insights about how to improve veterans' health care experiences. Veterans suggested that consideration of their previous military service would improve care. They lamented that the lack of military consciousness is a barrier to care. Finally, they suggested that clinicians pay close attention to the transition from service member to civilian, as reintegration to civilian life is a critical life experience. Training in veteran-centred care has not been a clear focus of medical education DISCUSSION: Veteran-centred care ensures optimal health care through ease of access to services, and through positive patient-provider interactions. Being aware of military culture can help providers to contextualise veterans' experiences and beliefs about health care seeking and illness management, particularly for invisible wounds of war, including traumatic brain injury (TBI) and post-traumatic stress disorder (PTSD).

The influence of MCAT and GPA preadmission academic metrics on interview scores

Medical school admissions interviews are used to assess applicants' nonacademic characteristics as advocated by the Association of American Medical Colleges' Advancing Holistic Review Initiative. The objective of this study is to determine whether academic metrics continue to significantly influence interviewers' scores in holistic processes by blinding interviewers to applicants' undergraduate grade point averages (uGPA) and Medical College Admission Test (MCAT). This study examines academic and demographic predictors of interview scores for two applicant cohorts at the University of Michigan Medical School. In 2012, interviewers were provided applicants' uGPA and MCAT scores; in 2013, these academic metrics were withheld from interviewers' files. Hierarchical regression analysis was conducted to examine the influence of academic and demographic variables on overall cohort interview scores. When interviewers were provided uGPA and MCAT scores, academic metrics explained more variation in interview scores (7.9%) than when interviewers were blinded to these metrics (4.1%). Further analysis showed a statistically significant interaction between cohort and uGPA, indicating that the association between uGPA and interview scores was significantly stronger for the 2012 unblinded cohort compared to the 2013 blinded cohort (β = .573, P < .05). By contrast, MCAT scores had no interactive effects on interviewer scores. While MCAT scores accounted for some variation in interview scores for both cohorts, only access to uGPA significantly influenced interviewers' scores when looking at interaction effects. Withholding academic metrics from interviewers' files may promote assessment of nonacademic characteristics independently from academic metrics.

Value of Near-Peer Mentorship from Protégé and Mentor Perspectives: A Strategy to Increase Physician Workforce Diversity

PURPOSE

Mentorship is a critical aspect of personal and professional development in academic medicine and helps to improve career satisfaction, productivity, and social networking. However, individuals from communities underrepresented in medicine (URiM) across the training continuum experience difficulty obtaining mentors, even prior to college. The value of near-peer mentorship is less well studied in medicine relative to other fields. The purpose of this mixed methods study is to explore the mentorship experiences of high school student protégés and their medical student mentors, as well as provide a description of the key features of the Doctors of Tomorrow (DOT) program.

METHODS

From November 2014 to September 2015, the authors used focus groups and critical incident narratives with 9th grade high school students as well as focus groups and semi-structured interviews with medical students to examine mentor-protégé experiences in the Doctors of Tomorrow (DOT) program. In 2016, thirty-one medical student mentors were asked to complete an online survey about their mentor experiences. Focus group and interview data were audio-recorded and transcribed verbatim. All data were coded using thematic analysis and recurring codes were organized into categories, then compared, scrutinized and arranged into broader themes by all authors.

RESULTS

The analysis of data from 70 medical students and 52 high school students revealed that mentors and protégés valued their mentor relationships based on regular in-person and electronic contact, shared common non-academic interests, and the anticipated prolonged nature of the relationship. Mentors also reported they initiated contact with their protégés every 2-3 weeks and monthly outside of program events, with email communication as the most common modality.

CONCLUSIONS

Near-peer relationships between high school and medical students may be an innovative strategy to promote health care careers, increase access to mentorship and develop meaningful mentorship relationships for URiM high school students.

Where Do Soldiers Really Come From? A Faculty Development Workshop on Veteran-Centered Care

PROBLEM

Addressing the medical concerns of veterans in both civilian health care systems and the Veterans Affairs (VA) health care system, where staff are familiar with issues of military reintegration, remains difficult but is increasingly important.

APPROACH

In 2013, the authors developed and implemented a faculty development workshop for practicing clinicians using the documentary Where Soldiers Come From. The workshop included topics on unconscious bias, the service member trajectory, health care disparities, and strategies for overcoming barriers to treating veterans with posttraumatic stress disorder and traumatic brain injury. The workshop engaged faculty in the following active-learning techniques: images in education; trigger video; critical thinking and reflective writing; think-pair-share; and large-group discussion. The workshop has been conducted at three locations with 46 health care professionals.

OUTCOMES

Thirty-one of 37 (84%) participants who completed the workshop evaluation were VA employees. The evaluation results show 25/32 (78.1%) participants indicated the workshop activities changed their knowledge, attitudes, and/or skills; 22/34 (64.7%) stated they had a better understanding of how to develop a care plan for veterans; and 27/34 (79.4%) stated they gained a better understanding of how to prepare for issues around returning veterans.

NEXT STEPS

To address the issue of veteran-centered care education more broadly, the authors have developed a massive open online course for health professionals, using most of the content from this workshop, which will be offered in spring 2016. Another important next step will be to deliver this workshop to and collect evaluation data from non-VA providers.

Medical school handoff education improves postgraduate trainee performance and confidence

OBJECTIVES

Determine postgraduate first-year (PGY-1) trainees ability to perform patient care handoffs and associated medical school training.

METHODS

About 173 incoming PGY-1 trainees completed an OSCE handoff station and a survey eliciting their training and confidence in conducting handoffs. Independent t-tests compared OSCE performance of trainees who reported receiving handoff training to those who had not. Analysis of variance examined differences in performance based on prior handoff instruction and across levels of self-assessed abilities, with significance set at p<0.05.

RESULTS

About 35% of trainees reported receiving instruction and 51% reported receiving feedback about their handoff performance in medical school. Mean handoff performance score was 69.5%. Trainees who received instruction or feedback during medical school had higher total and component handoff performance scores (p<0.05); they were also more confident in their handoff abilities (p<0.001). Trainees with higher self-assessed skills and preparedness performed better on the OSCE (p<0.05).

CONCLUSIONS

This study provides evidence that incoming trainees are not well prepared to perform handoffs. However, those who received instruction during medical school perform better and are more confident on standardized performance assessments. Given communication failures lead to uncertainty in patient care and increases in medical errors, medical schools should incorporate handoff training as required instruction.

Understanding the medical marriage: physicians and their partners share strategies for success

PURPOSE

Physicians and their spouses experience challenges to their relationships, some of which are shared with the general population and others of which are unique to the field of medicine. Trainees and junior faculty members remain curious about how they will balance their careers alongside marriage and family obligations. This study explores the challenges and strengths of dual- and single-physician relationships.

METHOD

In 2009, using appreciative inquiry as a theoretical framework, the authors conducted in-depth qualitative interviews with 25 individuals: 12 women and 13 men; 10 from dual-physician and 15 from single-physician relationships. A phenomenological analytic approach was used to arrive at the final themes.

RESULTS

Four themes emerged during the interviews: "We rely on mutual support in our relationships," "We recognize the important roles of each family member," "We have shared values," and "We acknowledge the benefit of being a physician to our relationships."

CONCLUSIONS

These findings illustrate that physicians identify strategies to navigate the difficult aspects of their lives. Learn ing from others' best practices can assist in managing personal relationships and work-life balance. These data can also be useful when counseling physicians on successful relationship strategies. As systems are developed that improve wellness and focus on role models for work-life balance, it will be important for this topic to be integrated into formal curricula across the continuum of medical education.

A successful faculty development program for implementing a sociocultural ePortfolio assessment tool

Portfolios are emerging as a tool for documenting learning progression and assessing competency. ePortfolios are appealing as a portable and fluid means of documenting both learning and relevant experiences in a large number of students. Competence and learning can be especially difficult to document in important aspects of education and training, such as patient-centeredness, the cultural context of disease, and social determinants of health that do not lend themselves to fact-based assessment methods. Successful implementation of a method such as an ePortfolio requires explicit faculty development, as many faculty members have limited expertise with modern educational assessment technology. As part of the authors' introduction of a Sociocultural ePortfolio Assessment Tool in the undergraduate medical curriculum, three faculty development workshops were held to expand faculty skills in using this technology. In addition to gaining comfort using a new Web-based technology, faculty members also needed to develop skills with providing mentored feedback and stimulating student reflection. Workshops were modeled after other successful programs reported in the literature and allowed faculty to develop a structured format for evaluating student content. Faculty members were given multiple opportunities to practice their newly developed skills providing mentored reflections using an ePortfolio. The workshop evaluations were positive, suggesting that faculty participation in the workshops were a necessary component for them to develop sufficient assessment skills for providing mentored reflection. Faculty members who participated in this program-whether or not they had content expertise in sociocultural medicine-valued the hands-on faculty development program.

Using artistic-narrative to stimulate reflection on physician bias

BACKGROUND

Physician bias toward patients directly impacts patient care and health outcomes. However, too little research has been done investigating avenues to bring about self-awareness in this area to eliminate commonly held stereotypes that fuel physician bias.

PURPOSES

The purpose of this study was to explore the ways in which 2nd-year medical students' reflected on an artistic-narrative presentation given by a woman with sickle cell disease.

METHODS

A total of 320 2nd-year medical student essays were reviewed for content relevant to the artistic-narrative presentation. A total of 75 essays were identified and served as the data for this study. These 75 essays were analyzed using qualitative interpretive thematic content analysis to identify students' perceptions and reflections on culture in the healthcare environment and the patient-provider relationship.

RESULTS

The analysis of the reflective essays revealed that this exercise helped students acknowledge physician bias in pain treatment, foster empathetic views toward patients as individuals, and recognize various ways in which biased beliefs can provide incite in healthcare disparities.

CONCLUSIONS

These findings suggest that the combination of methods--art, narrative, and written reflection--helped students acknowledge their own bias as well as the ways in which taken-for-granted assumptions and biases can influence patient care.

Impact of repeated health behavior counseling on women portraying an overweight standardized patient

OBJECTIVE

To investigate the influence of participation in a health behavior counseling (HBC) case by examining standardized patients' real-life attitudes and behaviors toward nutrition and physical activity habits.

METHODS

Focus group sessions were held with 10 standardized patients who regularly participated in the HBC case. Emergent themes were identified using grounded theory data analysis.

RESULTS

Standardized patients reported how participation with role playing in the HBC case prompted personal lifestyle changes. Changes occurred through increased awareness and motivation to make adjustments to health habits, such as exercise and improved food choices.

CONCLUSIONS AND IMPLICATIONS

The HBC model can be an effective method for helping patients to recognize the need to take action and implement positive lifestyle changes.

Trainees' perceptions of patient safety practices: recounting failures of supervision

BACKGROUND

Ensuring that trainees receive appropriate clinical supervision is one proven method for improving patient safety outcomes. Yet, supervision is difficult to monitor, even more so during advanced levels of training. The manner in which trainees' perceived failures of supervision influenced patient safety practices across disciplines and various levels of training was investigated.

METHODS

A brief, open-ended questionnaire, administered to 334 newly hired interns, residents, and fellows, asked for descriptions of situations in which they witnessed a failure of supervision and their corresponding response.

RESULTS

Of the 265 trainees completing the survey, 73 (27.5%) indicated having witnessed a failure of supervision. The analysis of these responses revealed three types of supervision failures-monitoring, guidance, and feedback. The necessity of adequate supervision and its accompanying consequences were also highlighted in the participants responses.

CONCLUSIONS

The findings of this study identify two primary sources of failures of supervision: supervisors' failure to respond to trainees' seeking of guidance or clinical support and trainees' failure to seek such support. The findings suggest that the learning environment's influence was sufficient to cause trainees to value their appearance to superiors more than safe patient care, suggesting that trainees' feelings may supersede patients' needs and jeopardize optimal treatment. The literature on the impact of disruptive behavior on patient care may also improve understanding of how intimidating and abusive behavior stifles effective communication and trainees' ability to provide optimal patient care. Improved supervision and communication within the medical hierarchy should not only create more productive learning environments but also improve patient safety.

Mainstreaming risk management education into new resident and fellow orientation

INTRODUCTION

Understanding patient safety events and causative factors is an important step in reducing preventable adverse events. The University of Michigan's Graduate Medical Education (GME) Office, Department of Risk Management (DRM), and Office of Clinical Affairs (OCA) collaborated to incorporate a video workshop as a formal introduction to patient safety during orientation for new residents and fellows. This workshop reinforced the importance of effective communication and supervision in patient safety.

METHODS

DRM and OCA produced a video depicting an actual, unanticipated outcome that resulted from a constellation of preventable circumstances, which allows the audience to observe communication and supervision issues that lead to a patient death. The video is followed by a discussion of the patient safety issues seen, why they occurred, and strategies for improvement. Trainee perceptions of the value of the experience were surveyed and collected using a qualitative survey.

RESULTS

Most responders found the video workshop helpful. Trainees perceived the video and facilitated discussion as an effective way to identify patient safety issues, available resources, and the culture of patient safety at the institution.

CONCLUSION

Trainee comments supported the video workshop as an effective way to highlight the importance of communication and supervision in relation to patient safety. In the future, the DRM, OCA, and GME hope to reinforce this shared vision of patient safety through combined educational efforts.

A validated search assessment tool: assessing practice-based learning and improvement in a residency program

OBJECTIVE

The objective of this study was to validate an assessment instrument for MEDLINE search strategies at an academic medical center.

METHOD

Two approaches were used to investigate if the search assessment tool could capture performance differences in search strategy construction. First, data from an evaluation of MEDLINE searches from a pediatric resident's longitudinal assessment were investigated. Second, a cross-section of search strategies from residents in one incoming class was compared with strategies of residents graduating a year later. MEDLINE search strategies formulated by faculty who had been identified as having search expertise were used as a gold standard comparison. Participants were presented with a clinical scenario and asked to identify the search question and conduct a MEDLINE search. Two librarians rated the blinded search strategies.

RESULTS

Search strategy scores were significantly higher for residents who received training than the comparison group with no training. There was no significant difference in search strategy scores between senior residents who received training and faculty experts.

CONCLUSION

The results provide evidence for the validity of the instrument to evaluate MEDLINE search strategies. This assessment tool can measure improvements in information-seeking skills and provide data to fulfill Accreditation Council for Graduate Medical Education competencies.

Using film in multicultural and social justice faculty development: scenes from Crash

We designed a faculty development workshop integrating scene excerpts from the Academy Award-winning movie Crash and active learning methods to encourage faculty participation and generate participant dialogue. The aims of this workshop were to enhance awareness of issues related to teaching in a multicultural classroom; stimulate discussion on teaching and learning about potentially contentious issues linked to race, ethnicity, religion, gender, geographical origin, and class; and expose faculty to the use of multimedia to facilitate discussion on topics of diversity and social justice. Twenty-five faculty attended 3 workshops in various venues, 18 of whom completed workshop evaluations. The workshop evaluation revealed that all participants believed that the scene excerpts and discussions helped them to reflect on their own attitudes toward race and diversity and felt better prepared to effectively facilitate classroom discussions on similar issues. This workshop is a useful tool for helping faculty to develop the skills and confidence to facilitate, manage, and stimulate discussions on controversial issues in multicultural education that may otherwise be avoided due to lack of expertise or experience.

Evidence for increasing diversity in graduate medical education: the competence of underrepresented minority residents measured by an intern objective structured clinical examination

OBJECTIVE

Some have commented that the limited number of underrepresented minorities (URMs) in United States' residency programs is due to a lack of qualified candidates. At the University of Michigan, an objective structured clinical examination is administered to incoming residents at the beginning of training to determine baseline competence. In this study we wanted to determine if competence differed for underrepresented minorities when compared to non-URM residents.

METHOD

The postgraduate orientation assessment, a 10-station examination, was developed that focused specifically on the knowledge and skills needed in the first 6 to 18 weeks of training. Stations assessed competence in informed consent, aseptic technique, evidence-based medicine, diagnostic images, critical laboratory values, cross-cultural communication, and Joint Commission requirements such as surgical fire safety, pain assessment, and management. We used various assessment measures including standardized patients, computer-based testing, and multiple-choice questions.

RESULTS

Our study found no significant differences in overall mean scores between URM residents and all other residents for the 5 years during which we administered the examination, except for 2002. This stands in contrast to the consistently worse performances of URM students on USMLE Step 1 and Step 2 Clinical Knowledge. Also, URM residents did not perform better or worse than their non-URM colleagues on standardized patient stations during the course of 5 years during which the examination was administered.

CONCLUSIONS

The postgraduate orientation assessment provides residency program directors with a standard format to measure initial clinical skills. When compared to incoming non-URM residents from a variety of medical schools, URM residents perform as well as other trainees. Our results may aid in the recruitment efforts of URM medical students into academic residency programs such as those at the University of Michigan.

An assessment tool for aseptic technique in resident physicians: a journey towards validation in the real world of limited supervision

PURPOSE

The purpose of this study was to describe the validation process for assessing an instrument to assess residents' aseptic technique skills.

METHODS

The validation study entailed comparisons of the performance of aseptic technique procedures between postgraduate year-1 (PGY-1) surgical residents and PGY-2/3 surgical residents. We also compared the performance of PGY-1 surgical residents from 2 different academic years for the same procedures. Finally, we compared the performance of novices (medical students) and experts (operating room nurses) in an effort to determine validity.

RESULTS

Our initial analysis found no significant difference between the performance of PGY-1 (mean score, 75.8) and PGY-2/3 (mean score, 75.6) surgical residents for aseptic technique (t((55))  =  0.84, P  =  0.404). Further investigation of validity was obtained to determine whether the no difference results reflected a lack of reliability or validity or a true equivalence between the 2 cohorts. The comparison of novices and experts produced the following findings. For reliability, the internal consistency of the checklist for each of the 2 raters was 0.87 and 0.71 (Cronbach α), interrater reliability was 0.74, with P < 0.001 (intraclass correlation coefficient) for the global scale. (Internal consistency was done within instrument, ie, between items not between raters.) For validity, operating room nurses outperformed students on the global scale (t(14)  =  7.47, P < 0.0001 and t((14))  =  10.66, P < 0.0001 for the 2 raters, respectively) and on several checklist items. The effect size values for raters were large (Cohen d  =  3.0 and 4.4), providing validity evidence for the ability of this assessment to detect difference in performance on this task.

CONCLUSION

The validation study showed that the instrument exhibited reliability and evidence for validity, making it useful for the assesment of aseptic technique skills in different specialties. Programs may want to consider using a validated instrument to check competence given that appropriate use of sterile technique frequently occurs in the context of unsupervised activities. Further work is needed to enhance resident skills in the area of aspectic technique because of limited improvement despite additional clinical experience.

Diabetes stories: use of patient narratives of diabetes to teach patient-centered care

A critical component to instituting compassionate, patient-centered diabetes care is the training of health care providers. Our institution developed the Family Centered Experience (FCE), a comprehensive 2-year preclinical program based on longitudinal conversations with patients about living with chronic illness. The goal of the FCE is to explore the experience of illness from the patient's perspective and ultimately to incorporate this perspective in clinical practice. In this qualitative study, we wished to investigate the impact of "diabetes stories"--the stories of FCE volunteers with diabetes--on medical students' understanding of diabetes and its management. Individual interviews were conducted with medical students who had worked with a volunteer with diabetes to answer the questions: "in what ways was learning through these 'diabetes stories' different from that acquired through lectures and textbooks," and "how did these stories impact the students' understanding of diabetes and its care?" Thematic analysis of the transcribed interviews was performed using Grounded Theory. Several major themes emerged: There was more to diabetes than the "scientific" knowledge acquired through lectures; the stories challenged students' assumptions about having or working with people with diabetes and allowed students to see the world through the perspective of someone with diabetes, and the stories motivated students' development as physicians and influenced their general perspectives of doctoring and medicine. First-person narratives of living with diabetes allow for learning in affective, experiential, and cognitive dimensions, stimulate self reflection and perspective-taking, and enhance growth through the challenging of previous assumptions, beliefs, and perspectives. This type of learning is transformative and may result in a shift in students' perspectives towards more open, inclusive attitudes towards patient-centered diabetes care.

A qualitative exploration of how the conflict between the formal and informal curriculum influences student values and behaviors

PURPOSE

The third-year students at one medical school told the authors that values core to patient-centered care were impossible to practice in clerkships, in a culture where supervisors role modeled behaviors in direct conflict with patient-centered care. As they developed a new medical student curriculum, the authors designed the Family Centered Experience (FCE) to help students achieve developmental goals and understand the importance of and provide a foundation for patient-centered care.

METHOD

The authors solicited members of the first cohort to complete the FCE (the class of 2007) to participate in this focus-group-based study halfway through the third year. They explored the influence of the FCE on students' experiences in the third-year clerkships, and how conflicts between the two learning experiences shaped these students' values and behaviors.

RESULTS

Students reported that during clerkships they experienced strong feelings of powerlessness and conflict between what they had learned about patient-centered care in the first two years and what they saw role modeled in the third year. Based on students' comments, the authors categorized students into one of three groups: those whose patient-centered values were maintained, compromised, or transformed.

CONCLUSIONS

Students revealed that their conflict was connected to feelings of powerlessness, along with exacerbating factors including limited time, concerns about expectations for their behavior, and pessimism about change. Role modeling had a significant influence on consequences related to students' patient-centered values.

Remediating students' failed OSCE performances at one school: the effects of self-assessment, reflection, and feedback

PURPOSE

To investigate whether and how use of an online remediation system requiring reflective review of performance and self-assessment influenced students' performance on objective structured clinical examination (OSCE) station repeats (subsequent to failure on the first attempt) and their self-assessments of their performance (between the first and second attempts).

METHOD

Fourth-year medical students' performances on seven OSCE stations were videotaped at University of Michigan Medical School in 2006. Failing students took the exam again; remediation included self-assessment and review, plus faculty guidance for failures that were greater than one standard error of measurement of the distribution. A total of 1,171 possible observations of students' actual performance and performance self-assessments were analyzed using independent and dependent t tests and within-subjects ANOVA.

RESULTS

Results indicate statistically significant changes in students' performance between first and second attempts and statistically significant improvements in self-assessment between first and second attempts. No significant changes were found between self-assessed and faculty-guided remediation.

CONCLUSIONS

This study provides evidence that OSCE remediation combining review, reflection, and self-assessment has a salutary effect on (subsequent) performance and self-assessment of performance.

Assessing the assessment: are senior summative OSCEs measuring advanced knowledge, skills, and attitudes?

PURPOSE

The authors investigated adaptation of Bloom's and Simpson's taxonomies for the medical (student) setting, and using the adapted taxonomies to determine whether a summative objective structured clinical examination (OSCE) used at their medical school was assessing higher-order knowledge, skills, and attitudes.

METHOD

Two faculty members (including H.M.H.) adapted the taxonomies and used them to categorize (knowledge, skills, or attitudes) and rank (by level within the taxonomies) every item on every OSCE station checklist. Interrater reliability was moderate to high.

RESULTS

Although there was a range of domains and levels within and across stations, on average every OSCE station was assessing learning behaviors at a lower level than expectations articulated in the school's goals for medical students' education.

CONCLUSIONS

The adapted taxonomies were useful for assessing the domains and levels of behaviors measured on the summative OSCE, and they can also be used to modify existing checklists or to create new assessment instruments that meet the expectations articulated in a school's goals for medical students' education.

The impact of facilitation of small-group discussions of psychosocial topics in medicine on faculty growth and development

PURPOSE

To use qualitative analysis of interview transcripts with clinician-educators who facilitate small-group discussions on psychosocial themes--including doctoring--to answer the question, "What impact does facilitating small-group discussions of the patient's experience with chronic illness, the doctor-patient relationship, and doctoring have on faculty instructors' attitudes regarding their roles as clinicians and teachers?"

METHOD

In 2006, in-depth, face-to-face interviews using an open-ended question format were conducted with individual faculty small-group instructors teaching in the Family Centered Experience and Longitudinal Case Studies courses at the University of Michigan Medical School. Interview transcripts were analyzed using grounded theory methodology to identify emerging themes. Accuracy of interpretations and saturation of themes was confirmed by repeated contextual reading of the transcripts.

RESULTS

Several major thematic codes emerged from the data. Facilitation of small-group discussions of psychosocial topics and doctoring fostered reflective approaches to patient care and teaching; enhanced interpersonal relationships between facilitators and their students, colleagues, and patients; and acted as a source of fulfillment and renewal among faculty facilitators.

CONCLUSIONS

Small-group teaching of the art of doctoring may stimulate personal and professional growth among faculty facilitators and renewed interest in teaching and patient care.

Use of interactive theater for faculty development in multicultural medical education

BACKGROUND

The development of critical consciousness, anchored in principles of social justice, is an essential component of medical education.

AIM

In order to assist faculty instructors in facilitating small-group discussions on potentially contentious issues involving race, gender, sexual orientation, and socioeconomic class, a faculty development workshop was created.

METHODS

The workshop used 'Forum Theater' techniques in which the audience was directly involved in determining the course of a simulated classroom discussion and conflict. We assessed the workshop's impact on the instructors' attitudes regarding facilitation of small-group discussions through two surveys: one to gauge immediate impressions, and another, 9-15 months later, to assess impact over time.

RESULTS

Immediately after the workshop, participants reported that the topics covered in the sketch and in the discussion were highly relevant. In the follow-up survey, the instructors agreed that the workshop had raised their awareness of the classroom experiences of minorities and women and had offered strategies for addressing destructive classroom dynamics. 72% reported that the workshop led to changes in their behavior as facilitators. Differences in responses according to gender were observed.

CONCLUSIONS

A workshop using interactive theater was effective in training faculty to facilitate small-group discussions about multicultural issues. This approach emphasizes and models the need to foster critical consciousness in medical education.