A Narrative Study of Equity in Clinical Assessment Through the Antideficit Lens

The Role of Faculty Development in Advancing Change at the Organizational Level

ABSTRACT

Although the traditional goal of faculty development (FD) has been to enhance individual growth and development, this goal may no longer suffice to address the compelling challenges faculty members are facing, such as increasing workloads, emotional well-being, and institutional support for education. Addressing these challenges will require change at the organizational level. The purpose of this perspective is to articulate a vision for FD programming that describes ways in which FD leaders, together with other educational leaders, can bring about change at the organizational level to support excellence and innovation in health professions education. To impact the organization at large, the authors propose a model that includes 4 major goals: (1) promoting individual and group development, through educational and leadership development programs, coaching and mentoring, and advanced degrees; (2) advocating for infrastructure and resources, including academies of medical educators, educational scholarship units, educational awards, and intramural funding for educational innovation and scholarship; (3) influencing policies and procedures, by engaging educators on key committees, reviewing appointment and promotion criteria, defining educator roles and portfolios, and valuing diversity, equity, and inclusion; and (4) contributing to organization-wide initiatives, such as addressing "hot button" issues, identifying value factors that support investments in FD and medical education, and enhancing the visibility of educators. In this model, the 4 goals are dynamically interconnected and can impact the culture of the organization. For each goal, the authors offer evidence-informed actions that FD leaders, along with other educational leaders, can adopt to improve the organizational culture and inspire institutionally relevant actions. Because each institution is unique, the options are illustrative and not prescriptive. The intent is to provide examples of how FD leaders and programs can enhance the educational mission through broader engagement with their institutions.

Inequity is woven into the fabric: a discourse analysis of assessment in pediatric residency training

Intrinsic inequity in assessment refers to sources of harmful discrimination inherent in the design of assessment tools and systems. This study seeks to understand intrinsic inequity in assessment systems by studying assessment policies and associated procedures in residency training, using general pediatrics as a discourse case study. Foucauldian discourse analysis (FDA) was conducted on assessment policy and procedure documents. Two authors independently prepared structured analytic notes using guiding questions. Documents and respective analytic notes were subsequently reviewed independently by all authors. Each author prepared further unstructured analytic notes on the documents' discourse. The authors then compared notes and constructed truth statements (i.e., interpretations of what the discourse establishes as true about the construct under study) and sub-strands (i.e., themes) that were repeated and legitimized across the documents via iterative discussion. Based on analysis, the authors constructed two truth statements. These truth statements, "good assessment is equitable assessment," and "everyone is responsible for inequity," conceptualized inequity in assessment as an isolated or individual-level aberration in an otherwise effective or neutral system. Closer examination of the truth statements and sub-strands in the discourse presented an alternative view, suggesting that inequity may in fact not be an aberration but rather an inherent feature of assessment systems.

URM: Underrepresented or Underrecognized? A Case Study of Black Pediatric Critical Care Physicians

PURPOSE

This study explored Black physicians' experience via an antideficit lens to gain new ideas for advancing minoritized physicians in academic medicine more broadly. Increasingly, systemic racism in academic medicine is intentionally acknowledged and named. However, many solutions to tackle racism and the overall paucity of Black physicians use a deficit framing, painting Black physicians and trainees as lacking preparation, interest, or experience and qualifications. Such solutions aim to help Black people assimilate into the "White Space" of academic medicine, rather than focusing on Black people's strengths.

METHOD

This qualitative study included 15 Black physicians and trainees in pediatric critical care medicine (PCCM) from across the country who participated in semistructured interviews. Through an antideficit lens, the researchers examined the social, cultural, and structural contexts influencing the participants' individual experiences. They analyzed the data combining thematic and narrative qualitative analysis approaches, including restorying.

RESULTS

The data help promote understanding of the landscape and context in which Black PCCM physicians become successful. Achievement took on different forms for the participants. Participants described enablers of achievement that supported them through their individual journeys spanning 3 general domains-intrinsic, interpersonal, and systemic. Three additional enablers were tied specifically to participants' Black identities-harnessing Blackness as a superpower, leaning in to lead, and successfully navigating the "unwritten rules."

CONCLUSIONS

By using an antideficit framework, this study delineates and centers participants' ingenuity in cultivating repertoires of practice that enabled them to succeed, despite challenges rooted in systemic racism. Going forward, rather than focus solely on what is missing, academic medicine should try to shift systems and regularly recognize and value the knowledge, expertise, and merit Black that physicians bring. Perhaps an appropriate framing is not that Black physicians are underrepresented in medicine; maybe instead, it is that they are underrecognized.

Developing Evidence for Equitable Assessment Characteristics Based on Clinical Learner Preferences Using Discrete Choice Experiments

PURPOSE

Medical education is only beginning to explore the factors that contribute to equitable assessment in clinical settings. Increasing knowledge about equitable assessment ensures a quality medical education experience that produces an excellent, diverse physician workforce equipped to address the health care disparities facing patients and communities. Through the lens of the Anti-Deficit Achievement framework, the authors aimed to obtain evidence for a model for equitable assessment in clinical training.

METHOD

A discrete choice experiment approach was used which included an instrument with 6 attributes each at 2 levels to reveal learner preferences for the inclusion of each attribute in equitable assessment. Self-identified underrepresented in medicine (UIM) and not underrepresented in medicine (non-UIM) (N = 306) fourth-year medical students and senior residents in medicine, pediatrics, and surgery at 9 institutions across the United States completed the instrument. A mixed-effects logit model was used to determine attributes learners valued most.

RESULTS

Participants valued the inclusion of all assessment attributes provided except for peer comparison. The most valued attribute of an equitable assessment was how learner identity, background, and trajectory were appreciated by clinical supervisors. The next most valued attributes were assessment of growth, supervisor bias training, narrative assessments, and assessment of learner's patient care, with participants willing to trade off any of the attributes to get several others. There were no significant differences in value placed on assessment attributes between UIM and non-UIM learners. Residents valued clinical supervisors valuing learner identity, background, and trajectory and clinical supervisor bias training more so than medical students.

CONCLUSIONS

This study offers support for the components of an antideficit-focused model for equity in assessment and informs efforts to promote UIM learner success and guide equity, diversity, and inclusion initiatives in medical education.

Academic Leadership Academy Summer Program: Clerkship Transition Preparation for Underrepresented in Medicine Medical Students

PROBLEM

Enhancing workforce diversity by increasing the recruitment of students who have been historically excluded/underrepresented in medicine (UIM) is critical to addressing healthcare inequities. However, these efforts are inadequate when undertaken without also supporting students' success. The transition to clerkships is an important and often difficult to navigate inflection point in medical training where attention to the specific needs of UIM students is critical.

INTERVENTION

We describe the design, delivery, and three-year evaluation outcomes of a strengths-based program for UIM second year medical students. The program emphasizes three content areas: clinical presentations/clinical reasoning, community building, and surfacing the hidden curriculum. Students are taught and mentored by faculty, residents, and senior students from UIM backgrounds, creating a supportive space for learning.

CONTEXT

The program is offered to all UIM medical students; the centerpiece of the program is an intensive four-day curriculum just before the start of students' second year. Program evaluation with participant focus groups utilized an anti-deficit approach by looking to students as experts in their own learning. During focus groups mid-way through clerkships, students reflected on the program and identified which elements were most helpful to their clerkship transition as well as areas for programmatic improvement.

IMPACT

Students valued key clinical skills learning prior to clerkships, anticipatory guidance on the professional landscape, solidarity and learning with other UIM students and faculty, and the creation of a community of peers. Students noted increased confidence, self-efficacy and comfort when starting clerkships.

LESSONS LEARNED

There is power in learning in a community connected by shared identities and grounded in the strengths of UIM learners, particularly when discussing aspects of the hidden curriculum in clerkships and sharing specific challenges and strategies for success relevant to UIM learners. We learned that while students found unique benefits to preparing for clerkships in a community of UIM students, near peers, and faculty, future programs could be enhanced by pairing this formal intensive curriculum with more longitudinal opportunities for community building, mentoring, and career guidance.

Is it Time for Competency-Based Education to Move Forward in Pharmacy Education?

Competency-based education is rapidly emerging as a paradigmatic shift in health professions education, as we grapple with the realities of ever-changing and increasing demands of society and health systems. While pharmacy educators are becoming more familiar with this paradigm, colleagues in medical education have been exploring models and methods of competency-based education for many years, and their experiences can be illuminating for us. The persistent question that drives continuous quality improvement in pharmacy education and the development of initiatives within American Association of Colleges of Pharmacy might be stated as "Is there a better (more effective, more efficient) way to prepare pharmacists (future and current) to meet the medication-related needs of the public?"

How medical learners and educators decide what counts as mistreatment: A qualitative study

INTRODUCTION

The mistreatment or abuse (maltreatment) of medical learners by their peers and supervisors has been documented globally for decades, and there is significant research about the prevalence, sequelae and strategies for intervention. However, there is evidence that learners experience maltreatment as being less clear cut than do researchers, educators and administrators. This definitional ambiguity creates problems for understanding and addressing this issue. The objective of this study was to understand how medical learners and educators make sense of less-than-ideal interactions in the clinical learning environment, and to describe which factors influenced their perception that the encounter constituted maltreatment.

METHODS

Using constructivist grounded theory, we interviewed 16 medical students, 15 residents or fellows, and 18 educators associated with a single medical school (n = 49). Data collection began with the most junior learners, iterating with analysis as we progressed through the project. Constant comparative analysis was used to gather and compare stories of 'definitely', 'maybe' and 'definitely not' maltreatment across a variety of axes including experience level, clinical setting and type of interaction.

RESULTS

Our data show that learners and educators have difficulty classifying their experiences of negative interpersonal interaction, except in the most severe and concrete cases. While there was tremendous variation in the way they categorised similar experiences, there was consistency in the elements drawn upon to make sense of those experiences. Participants interpreted negative interpersonal interactions on an individual basis by considering factors related to the interaction, initiator and recipient.

CONCLUSIONS

Only the most negative behaviour is consistently understood as maltreatment; a complex process of individual sense-making is required to determine the acceptability of each interaction. The differences between how individuals judge these interactions highlight an opportunity for administrative, research and faculty development intervention.

Student and Teacher Perspectives on Equity in Clinical Feedback: A Qualitative Study Using a Critical Race Theory Lens

PURPOSE

The authors aimed to gain a better understanding of students' and teachers' perspectives about whether clinical clerkship feedback is provided equitably irrespective of a student's race/ethnicity.

METHOD

A secondary analysis of existing interview data was conducted, focusing on racial/ethnic disparities in clinical grading. Data had been acquired from 29 students and 30 teachers at 3 U.S. medical schools. The authors performed secondary coding on all 59 transcripts, writing memos focused on statements related to aspects of feedback equity and developing a template for coding students' and teachers' observations and descriptions specific to clinical feedback. Using the template, memos were coded, and thematic categories emerged describing perspectives on clinical feedback.

RESULTS

Forty-eight (22 teachers and 26 students) participants' transcripts provided narratives about feedback. Both student and teacher narratives described how students who are racially/ethnically underrepresented in medicine may receive less helpful formative clinical feedback needed for professional development. Thematic analysis of narratives yielded 3 themes related to feedback inequities: 1) teachers' racial/ethnic biases influence the feedback they provide students, 2) teachers have limited skill sets to provide equitable feedback, and 3) racial/ethnic inequities in the clinical learning environment shape clinical and feedback experiences.

CONCLUSIONS

Narratives indicated that both students and teachers perceived racial/ethnic inequities in clinical feedback. Teacher- and learning environment-related factors influenced these racial/ethnic inequities. These results can inform medical education's efforts to mitigate biases in the learning environment and provide equitable feedback to ensure every student has what they need to develop into the competent physician they aspire to be.

"My Assessments Are Biased!" Measurement and Sociocultural Approaches to Achieve Fairness in Assessment in Medical Education

Assessing learners is foundational to their training and developmental growth throughout the medical education continuum. However, growing evidence shows the prevalence and impact of harmful bias in assessments in medical education, accelerating the urgency to identify solutions. Assessment bias presents a critical problem for all stages of learning and the broader educational system. Bias poses significant challenges to learners, disrupts the learning environment, and threatens the pathway and transition of learners into health professionals. While the topic of assessment bias has been examined within the context of measurement literature, limited guidance and solutions exist for learners in medical education, particularly in the clinical environment. This article presents an overview of assessment bias, focusing on clinical learners. A definition of bias and its manifestations in assessments are presented. Consequences of assessment bias are discussed within the contexts of validity and fairness and their impact on learners, patients/caregivers, and the broader field of medicine. Messick's unified validity framework is used to contextualize assessment bias; in addition, perspectives from sociocultural contexts are incorporated into the discussion to elaborate the nuanced implications in the clinical training environment. Discussions of these topics are conceptualized within the literature and the interventions used to date. The article concludes with practical recommendations to overcome bias and to develop an ideal assessment system. Recommendations address articulating values to guide assessment, designing assessment to foster learning and outcomes, attending to assessment procedures, promoting continuous quality improvement of assessment, and fostering equitable learning and assessment environments.

Strategies for Advancing Equity in Frontline Clinical Assessment

Educational equity in medicine cannot be achieved without addressing assessment bias. Assessment bias in health professions education is prevalent and has extensive implications for learners and, ultimately, the health care system. Medical schools and educators desire to minimize assessment bias, but there is no current consensus on effective approaches. Frontline teaching faculty have the opportunity to mitigate bias in clinical assessment in real time. Based on their experiences as educators, the authors created a case study about a student to illustrate ways bias affects learner assessment. In this paper, the authors use their case study to provide faculty with evidence-based approaches to mitigate bias and promote equity in clinical assessment. They focus on 3 components of equity in assessment: contextual equity, intrinsic equity, and instrumental equity. To address contextual equity, or the environment in which learners are assessed, the authors recommend building a learning environment that promotes equity and psychological safety, understanding the learners' contexts, and undertaking implicit bias training. Intrinsic equity, centered on the tools and practices used during assessment, can be promoted by using competency-based, structured assessment methods and employing frequent, direct observation to assess multiple domains. Instrumental equity, focused on communication and how assessments are used, includes specific, actionable feedback to support growth and use of competency-based narrative descriptors in assessments. Using these strategies, frontline clinical faculty members can actively promote equity in assessment and support the growth of a diverse health care workforce.

Racial/Ethnic Disparities in Clerkship Grading: Perspectives of Students and Teachers

PURPOSE

Racial/ethnic disparities exist in clinical clerkship grading, yet little is known about medical student and faculty perspectives on why these disparities occur. This study explored what happens during clerkships that might explain grading disparities.

METHOD

Medical students and clerkship teachers at 3 U.S. medical schools completed a demographic survey and semistructured interview. The constant comparative method was used to analyze transcripts by inductively developing codes; grouping codes in categories; and refining codes, descriptions, and group assignments to identify themes. Interpretations of and relationships among themes were iteratively discussed to develop a grounded theory.

RESULTS

Fifty-nine participants (29 medical students, 30 teachers [28 clinical faculty, 2 residents]) were interviewed in 2020. The Social Milieu of Medical Education (relationships, fit, opportunities, and judgments in the clinical-learning setting) was the organizing theme, influenced by 5 additional themes: Societal Influence (experiences in society), Students' Characteristics and Background (personal characteristics and experiences outside medical school), Assessment Processes (collection of student performance data and how data inform grades), Learning Environment (resources available and messaging within the clinical setting), and Students' Interactions and Reactions (interactions with and reactions to peers and teachers). The grounded theory highlights complex, multilayered aspects of how the social milieu of medical education is shaped by and shapes students' experiences, relationships, and clerkship assessments and promotes clerkship-grading disparities.

CONCLUSIONS

Mitigating clerkship-grading disparities will require intervening on interrelated, contextual factors to provide equitable opportunities for students from diverse backgrounds and with varying styles of engagement in clinical-learning settings, along with attending to modifying assessment processes.

Racial Bias on the Emergency Medicine Standardized Letter of Evaluation

BACKGROUND

Studies on components of residency applications have shown evidence of racial bias. The Standardized Letter of Evaluation (SLOE) is an assessment measure for emergency medicine (EM) residency applications and, as more specialties opt to use SLOEs in place of narrative letters of recommendation, understanding bias on standardized assessments is essential.

OBJECTIVE

To determine whether there is a difference in rankings on the EM SLOE between underrepresented in medicine (UIM) and non-UIM applicants, White and non-White applicants, and to examine whether differences persist after controlling for other characteristics.

METHODS

The sample was drawn from medical students who applied to EM residency at the study institution in 2019. We compared rankings between UIM and non-UIM students and between students of each individual race/ethnicity and White students, after controlling for United States Medical Licensing Examination Step scores, Alpha Omega Alpha status, type of school (US MD, US DO, internation medical graduate), Medical Student Performance Evaluation class percentile, affiliated program vs visiting clerkship SLOE, gender and the interaction of race/ethnicity and gender, and adjusted for students submitting multiple SLOEs, using ordinal regression.

RESULTS

There were 1555 applicants to the study institution in 2019; 1418 (91.2%) had a SLOE and self-identified race/ethnicity. After controlling for applicant characteristics, non-UIM students were significantly more likely to be ranked higher than UIM students on "Rank Against Peers," (OR 1.46, 95% CI 1.03-2.07) and Grade (OR 1.46, 95% CI 1.05-2.04).

CONCLUSIONS

Analysis of EM SLOEs submitted to our institution demonstrates racial bias on this standardized assessment tool, which persists after controlling for other performance predictors.

Growth, Engagement, and Belonging in the Clinical Learning Environment: the Role of Psychological Safety and the Work Ahead

Psychological safety is the perception that an environment is safe for interpersonal risk taking, exposing vulnerability, and contributing perspectives without fear of being shamed, blamed, or ignored. The presence of psychological safety has been associated with improved team learning and innovation, leader inclusivity, and team members' sense of belonging. In medical education, psychological safety has additional benefits: it allows learners to be present in the moment and to focus on the tasks at hand, and reduces trainee focus on image. Several key features of psychologically safe environments have already been described, including the presence of high-quality relationships, the absence of social positioning, a learner-driven and flexible learning agenda, the lack of formal assessment, and time for debriefing. However, many of the structures and cultural traditions in medical education are in clear opposition to these features. This paper describes the current barriers to psychological safety in medical education, and sets out an agenda for change. In accordance with benefits seen in other sectors, we anticipate that an emphasis on relationships and psychological safety will support the learning, inclusion, and success of medical trainees.

Pipeline to the Physician Assistant Profession: A Look to the Future

ABSTRACT

The current pipeline of physician assistant (PA) school applicants reflects the future workforce of the profession, which is why the admissions process with all its components and variables is so important. Many studies have shown that a workforce that represents the patients it cares for leads to improved health outcomes, especially among underrepresented minority populations. Yet, PA programs have made little progress over the past 2 decades in increasing the diversity of matriculants and graduates. As a profession, it is our collective responsibility to intentionally advance diversity, equity, and inclusion, and examining the admissions process would be the most logical place to start.

Professional identity formation for underrepresented in medicine learners

Professional identity formation (PIF) is a dynamic developmental process by which individuals merge the knowledge and skills of clinical practice with the values and behaviors of their personal identity. For an individual physician, this process is a continuum which begins with their nascent interest in the field of medicine and extends through the end stages of a medical career. The impact of PIF has become a growing focus of medical education research in the last decade, and in that time, little attention has been paid to the influence underrepresented in medicine (UIM) identities may have on this fundamental process. Importantly, in discussions of how medical educators can support and encourage successful PIF, there is little discussion on the distinct challenges and different needs UIM learners may have. The purpose of this paper is to address the current literature around PIF for UIM trainees. This review explores various threats to identity formation, including identity fusion, stereotype threat, minority tax, implicit bias, and lack of mentorship. Evidence-based strategies to mitigate these challenges is also presented, including furthering institutional support for PIF, building the community of practice, supporting an inclusive environment, and developing PIF assessment tools. Through exploring these challenges and solutions, we are better able to address the needs of UIM trainees and physicians as they proceed in their PIF during their lifelong journey in medicine.

Bias in assessment: name, reframe, and check in

Cognitive bias permeates almost every learner assessment in medical education. Assessment bias has the potential to affect a learner's education, future career and sense of self-worth. Decades of data show that there is little educators can do to overcome bias in learner assessments. Using in-group favouritism as an example, we offer an evidence-based, three-step solution to understand and move forward with cognitive bias in assessment: (1) Name: a simple admission about the presence of inherent bias in assessment, (2) Reframe: a rephrasing of assessment language to shed light on the assessor's subjectivity and (3) Check-in: a chance to ensure learner understanding and open lines of bidirectional communication. This process is theory-informed and based on decades of educational, sociological and psychological literature; we offer it as a logical first step towards a much-needed paradigm shift towards addressing bias in learner assessment.