1
|
Kelleher M, Kinnear B, Weber DE, Knopp MI, Schumacher D, Warm E. Point/counterpoint: Should we stop writing and reading letters of recommendation for residency selection? J Hosp Med 2024; 19:858-862. [PMID: 38923809 DOI: 10.1002/jhm.13440] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/15/2024] [Revised: 06/11/2024] [Accepted: 06/11/2024] [Indexed: 06/28/2024]
Affiliation(s)
- Matthew Kelleher
- Department of Pediatrics, Division of Hospital Medicine, Internal Medicine and Pediatrics Hospital Medicine, University of Cincinnati College of Medicine/Cincinnati Children's Hospital Medical Center, Cincinnati, Ohio, USA
| | - Benjamin Kinnear
- Department of Pediatrics, Division of Hospital Medicine, University of Cincinnati College of Medicine/Cincinnati Childrens Medical Center, Cincinnati, Ohio, USA
| | - Danielle E Weber
- Department of Pediatrics, Division of Hospital Medicine, University of Cincinnati College of Medicine/Cincinnati Childrens Medical Center, Cincinnati, Ohio, USA
| | - Michelle I Knopp
- Department of Pediatrics, Division of Hospital Medicine, University of Cincinnati College of Medicine/Cincinnati Childrens Medical Center, Cincinnati, Ohio, USA
| | - Daniel Schumacher
- Department of Pediatrics, Division of Emergency Medicine, University of Cincinnati College of Medicine/Cincinnati Childrens Medical Center, Cincinnati, Ohio, USA
| | - Eric Warm
- Department of Internal Medicine and Division of General Internal Medicine, University of Cincinnati College of Medicine, Cincinnati, Ohio, USA
| |
Collapse
|
2
|
Sundaram N, Walker KK, Cladis FP. Do certain phrases in letters of recommendation predict success in fellowship performance? MEDICAL TEACHER 2024; 46:978-981. [PMID: 38306959 DOI: 10.1080/0142159x.2024.2311268] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/28/2023] [Accepted: 01/24/2024] [Indexed: 02/04/2024]
Abstract
BACKGROUND Letters of recommendation (LORs) are a valued, yet imperfect tool. Program directors (PDs) score phrases such as give my highest recommendation and top 5 to 10% of students as positive. Although positive phrases are valued by PDs, there is no evidence that these phrases predict performance. We attempt to identify whether 12 specific phrases found in letters of recommendation predict future performance of fellows. METHODS LORs were evaluated for 12 select phrases and statements. Alpha Omega Alpha (AOA) status, Step 2 Clinical Knowledge (CK) score, and whether the letter writer was personally known to our admission's committee were also categorized. Logistic regressions were performed to evaluate the relationship of the independent variables with fellow performance. RESULTS Using multivariate logistic regression, one of the best residents (OR = 4.02, 95% CI (1.0, 15.9), p < 0.05), exceeds expectations (OR = 4.74, 95% CI (1.4, 16.3), p = 0.01), and give my highest recommendation (OR = 3.87, 95% CI (1.3, 11.7), p = 0.02) predicted positive performance. Highly recommend (OR = 0.31, 95% CI (0.1, 1.0), p < 0.05) and top 5 to 10% (OR = 0.05, 95% CI (0.0, 0.6), p = 0.02) predicted negative performance. The remaining phrases did not correlate to fellowship performance. CONCLUSION The current LOR evaluation process may place undo importance on phrases that have limited bearing on a candidate's success in training. Training both letter readers and writers to avoid using coded language or avoid assigning improper importance to select phrases may help improve the candidate selection process.
Collapse
Affiliation(s)
- Niteesh Sundaram
- Department of General Surgery, Crozer Health, Resident Physician Postgraduate Year 2, PA, USA
| | - K Karisa Walker
- Department of Anesthesiology, University of Florida, Gainesville, FL, USA
| | - Franklyn P Cladis
- Department of Anesthesiology and Perioperative Medicine, University of Pittsburgh Medical Center, Pittsburgh, PA, USA
| |
Collapse
|
3
|
Pianello SR, Abouezzi JM, Weber GM, Drugge E, Medow MS, Abramowicz AE. Do Program Directors of Anesthesiology Residency Programs Interpret Narrative Letters of Recommendation as Intended? Cureus 2024; 16:e63573. [PMID: 39087200 PMCID: PMC11290036 DOI: 10.7759/cureus.63573] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 07/01/2024] [Indexed: 08/02/2024] Open
Abstract
Background Letters of recommendation (LORs) are an important part of the application process for medical residency programs with most specialties preferring a narrative format. Given the inherent subjectivity of narrative LORs, the current study sought to determine whether the intended messages of narrative LORs written for applicants to anesthesiology residency programs are accurately interpreted by readers. Methodology Anonymous online surveys were sent via the Qualtrics platform to program directors (PDs) of the Accreditation Council for Graduate Medical Education-accredited anesthesiology residency programs in the Mid-Atlantic region as designated by the Electronic Residency Application Service, which consists of the states of New York, Pennsylvania, and New Jersey. Each PD participant received five surveys, each of which was attached to a de-identified LOR that was written by another PD located at an institution in the same region. Both the letter writer and study participants were asked to score LORs on a Likert-like scale. Participants were additionally asked whether the LORs, if received, would influence their decision to either offer an interview to the applicant or to rank the applicant. Finally, participants were asked to note any specific words or phrases within the LORs that they found to be particularly impactful. Results Overall, 10 of 34, 29.41%, PDs responded to the survey. There was a high correlation between the LOR intent and the respondents' assigned rating (Spearman's rho = 0.7973, p < 0.001). Responses were more accurate for "outstanding and excellent" LORs compared to the lower three categories. Results were unaffected after adjusting for respondents' years of experience as PDs. Additionally, 71.6% indicated that the LORs would influence the decision about offering an interview, and 56.5% stated that the LORs would influence a ranking decision. Conclusions Our results indicate that respondents' perception of LORs correlated strongly with the intent of the writer. Additionally, respondents seemed to value LORs for interview and ranking decisions.
Collapse
Affiliation(s)
| | | | - Garret M Weber
- Anesthesiology, Westchester Medical Center, Valhalla, USA
- Anesthesiology, New York Medical College, Valhalla, USA
| | - Elizabeth Drugge
- Public Health, New York Medical College School of Health Sciences and Practice, Valhalla, USA
| | - Marvin S Medow
- Pediatrics/Physiology, New York Medical College, Valhalla, USA
| | - Apolonia E Abramowicz
- Anesthesiology, Westchester Medical Center, Valhalla, USA
- Anesthesiology, New York Medical College, Valhalla, USA
| |
Collapse
|
4
|
Dotters-Katz SK, Grace MR, Pilliod R, Hofler LG, Temming L, Shanks A. Cracking the code: interpreting content and phrases used in maternal-fetal medicine fellowship letters of recommendation. Am J Obstet Gynecol MFM 2024; 6:101404. [PMID: 38871295 DOI: 10.1016/j.ajogmf.2024.101404] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/16/2024] [Accepted: 05/16/2024] [Indexed: 06/15/2024]
Abstract
BACKGROUND Letters of recommendation for Maternal-Fetal Medicine(MFM) fellowship are a critical part of the applicant selection process. However, data regarding best practices for how to write LOR for MFM is limited. Similarly, within letters of recommendation, differences in the "code" or meaning of summative words/phrases used at the end of letters of recommendation are seen between surgery, pediatrics, and medicine. However, data regarding code MFM Letters of recommendation are quite limited. OBJECTIVE We sought to describe what Maternal-Fetal Medicine program directors value in letters of recommendation for fellowship applicants and how PDs interpret commonly used summative words/phrases. STUDY DESIGN After IRB exemption, subject matter experts developed an e-survey querying the importance of various letters of recommendation "best practices" described by other specialties. Content and face validation were performed prior to dissemination. This cross-sectional survey was administered to MFM program directors in February 2023. The primary outcome was the relative importance of letters of recommendation content areas. Secondary outcomes included the strength of each summative "code" phrase. Descriptive analysis was performed and principal component analysis (PCA) was then used to reduce the list of phrases to their underlying dimensions. Statistical analysis was performed by SPSS 29.0. RESULTS Of 104 MFM program directors sent the survey, 70 (67%) responded. MFM program directors reviewed an average of 78 applications (SD, 30) with 60% writing ≥3 letters/year. Ninety-one percent of respondents noted that letters of recommendation are important/very important in shaping impressions of an applicant. Respondents reported the depth of interaction with an applicant, the applicant's specific behavior traits, the applicant's abilities and a summative statement including strength of the recommendation as important content for MFM fellowship letters of recommendation. Letter length, use of bold/italics, and restating the applicant's curriculum vitae were considered not important. Following PCA with varimax rotation, 14 specific phrases used in letters of recommendation were reduced to 5 themes: high qualitative assessments, average qualitative assessments, objective metrics, exceeding expectations and grit. These themes accounted for 64.6% of the variance in the model (KMO 0.7, Bartlett's Test of Sphericity p<.01). Phrases that respondents considered positive included: "Top 5%," "Want to keep," and "highest recommendation," (all mean score≥4.5/5), while "expected level," "showed improvement," and "2nd quartile" were negatively associated code words (all mean score <2.5/5). CONCLUSION MFM program directors reported that descriptions of an applicant's abilities, behavior traits, and depth of the writer's interactions with the applicant were all important components of an MFM fellowship letters of recommendation. Letter length, bold/italics, and highlights from the CV were not important. A clear "code" emerged regarding summative phrases included in letters of recommendation. Dissemination of these data might help less experienced letter writers send a clearer message and ensure all letter writers have a shared mental model.
Collapse
Affiliation(s)
- Sarah K Dotters-Katz
- Department of Obstetrics and Gynecology, Duke University, Durham NC (Dotters-Katz).
| | - Matthew R Grace
- Department of Obstetrics and Gynecology, Vanderbilt University, Nashville, TN (Grace)
| | - Rachel Pilliod
- Maternal Fetal Medicine, Alina Health, Minneapolis, MN (Pilliod)
| | - Lisa G Hofler
- Department of Obstetrics and Gynecology, University of New Mexico, Albuquerque, NM (Hofler)
| | - Lorene Temming
- Department of Obstetrics and Gynecology, Atrium Health Wake Forest, Charlotte, NC (Temming)
| | - Anthony Shanks
- Department of Obstetrics and Gynecology, Indiana University School of Medicine, Indianapolis, IN (Shanks)
| |
Collapse
|
5
|
Sehgal RR, Sarva H, Safdieh JE, Robbins MS. Pearls and pitfalls in letters of recommendation for neurology residency applications. J Neurol Sci 2024; 459:122951. [PMID: 38461761 DOI: 10.1016/j.jns.2024.122951] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/24/2023] [Revised: 02/28/2024] [Accepted: 03/04/2024] [Indexed: 03/12/2024]
Abstract
Letters of recommendation are a cornerstone of residency applications. Variability and bias in letters exists across specialties, neurology being no exception. Studies done in other specialty fields assessing nuanced language uncovered key attention points for improvement and mitigation of bias, lessons from which should be applied in the field of neurology. We review common pearls and pitfalls in the letter solicitation, writing and reading process, with suggested best-practices for residency applicants, letter writers, and program faculty reviewers. We advocate for the thoughtful selection of writers, emphasis on highlighting professional skills, and attention to implicit bias. This discussion focuses on recommendations for US advanced or categorical neurology programs, but elements of this guidance may apply more broadly to fellowship and faculty promotion letters as well.
Collapse
Affiliation(s)
- Ryka R Sehgal
- Department of Neurology, University of California - San Francisco, 505 Parnassus Avenue, M798, Box 0114, San Francisco, CA, USA 94143.
| | - Harini Sarva
- Department of Neurology, Weill Cornell Medicine/NewYork-Presbyterian Hospital, 520 E 70(th) St, Suite 607, New York City, NY, USA 10021.
| | - Joseph E Safdieh
- Department of Neurology, Weill Cornell Medicine/NewYork-Presbyterian Hospital, 520 E 70(th) St, Suite 607, New York City, NY, USA 10021.
| | - Matthew S Robbins
- Department of Neurology, Weill Cornell Medicine/NewYork-Presbyterian Hospital, 520 E 70(th) St, Suite 607, New York City, NY, USA 10021.
| |
Collapse
|
6
|
Dotters-Katz SK, Kirsch EP, Hofler LG, Temming L, Shanks A, Gray B. Letters of recommendation for maternal-fetal medicine fellowship: time for best practices. Am J Obstet Gynecol MFM 2023; 5:101075. [PMID: 37414343 DOI: 10.1016/j.ajogmf.2023.101075] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2023] [Revised: 06/22/2023] [Accepted: 06/28/2023] [Indexed: 07/08/2023]
Affiliation(s)
- Sarah K Dotters-Katz
- Department of Obstetrics and Gynecology, Duke University Medical Center, 2608 Erwin Rd. Ste. 220, Durham, NC 27705.
| | - Elayna P Kirsch
- Department of Obstetrics and Gynecology, Duke University Medical Center, 2608 Erwin Rd. Ste. 220, Durham, NC 27705
| | - Lisa G Hofler
- Department of Obstetrics and Gynecology, University of New Mexico, Albuquerque, NM
| | - Lorene Temming
- Department of Obstetrics and Gynecology, Wake Forest School of Medicine Atrium Health, Charlotte, NC
| | - Anthony Shanks
- Department of Obstetrics and Gynecology, Indiana University, Indianapolis, IN
| | - Beverly Gray
- Department of Obstetrics and Gynecology, Duke University, Durham, NC
| |
Collapse
|
7
|
Tavarez MM, Baghdassarian A, Bailey J, Caglar D, Eckerle M, Fang A, McVety K, Nagler J, Ngo TL, Rose JA, Roskind CG, Benedict FT, Nesiama JAO, Thomas AA, Langhan ML. A Call to Action for Standardizing Letters of Recommendation. J Grad Med Educ 2022; 14:642-646. [PMID: 36591418 PMCID: PMC9765898 DOI: 10.4300/jgme-d-22-00131.1] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/23/2022] Open
Affiliation(s)
- Melissa M. Tavarez
- Melissa M. Tavarez, MD, MS, is Program Director, Pediatric Emergency Medicine (PEM) Fellowship, Department of Pediatrics, Division of PEM, University of Pittsburgh School of Medicine/UPMC Children's Hospital of Pittsburgh
| | - Aline Baghdassarian
- Aline Baghdassarian, MD, MPH, is Program Director, PEM Fellowship, Department of Pediatrics, Section of PEM, Inova LJ Murphy Children's Hospital
| | - Jessica Bailey
- Jessica Bailey, MD, is Program Director, PEM Fellowship, Department of Emergency Medicine, Oregon Health & Science University
| | - Derya Caglar
- Derya Caglar, MD, is Program Director, PEM Fellowship, Department of Pediatrics, Division of PEM, University of Washington School of Medicine/Seattle Children's Hospital
| | - Michelle Eckerle
- Michelle Eckerle, MD, MPH, is Program Director, PEM Fellowship, Department of Pediatrics, University of Cincinnati College of Medicine/Division of PEM, Cincinnati Children's Hospital Medical Center
| | - Andrea Fang
- Andrea Fang, MD, is Clinical Associate Professor, Department of Emergency Medicine, Stanford University School of Medicine
| | - Katherine McVety
- Katherine McVety, MD, is Associate Program Director, PEM Fellowship, Department of Pediatrics, Division of PEM, Children's Hospital of Michigan
| | - Joshua Nagler
- Joshua Nagler, MD, MHPEd, is Program Director, PEM Fellowship, Department of Pediatrics, Division of PEM, Boston Children's Hospital
| | - Thuy L. Ngo
- Thuy L. Ngo, DO, MEd, is Program Director, PEM Fellowship, Department of Pediatrics, Division of PEM, Johns Hopkins University School of Medicine
| | - Jerri A. Rose
- Jerri A. Rose, MD, is Program Director, PEM Fellowship, Department of Pediatrics, Division of PEM, UH Rainbow Babies & Children's Hospital
| | - Cindy Ganis Roskind
- Cindy Ganis Roskind, MD, is Program Director, PEM Fellowship, Department of Emergency Medicine, Division of PEM, Columbia University Irving Medical Center
| | - Frances Turcotte Benedict
- Frances Turcotte Benedict, MD, MPH, is Program Director, PEM Fellowship, Department of Pediatrics, Division of PEM, University of Missouri Kansas City School of Medicine/Children's Mercy Hospital
| | - Jo-Ann O. Nesiama
- Jo-Ann O. Nesiama, MD, MS, is Associate Program Director, PEM Fellowship, Department of Pediatrics, Division of Emergency Medicine, UT Southwestern Medical Center
| | - Anita A. Thomas
- Anita A. Thomas, MD, MPH, is Associate Program Director, PEM Fellowship, Department of Pediatrics, Division of PEM, University of Washington School of Medicine/Seattle Children's Hospital
| | - Melissa L. Langhan
- Melissa L. Langhan, MD, MHS, is Program Director, PEM Fellowship, Department of Pediatrics, Section of PEM, Yale University School of Medicine
| |
Collapse
|
8
|
Dhaliwal G, Hauer KE. Excellence in medical training: developing talent-not sorting it. PERSPECTIVES ON MEDICAL EDUCATION 2021; 10:356-361. [PMID: 34415554 PMCID: PMC8377327 DOI: 10.1007/s40037-021-00678-5] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/24/2020] [Revised: 06/22/2021] [Accepted: 06/23/2021] [Indexed: 06/02/2023]
Abstract
Many medical schools have reconsidered or eliminated clerkship grades and honor society memberships. National testing organizations announced plans to eliminate numerical scoring for the United States Medical Licensing Examination Step 1 in favor of pass/fail results. These changes have led some faculty to wonder: "How will we recognize and reward excellence?" Excellence in undergraduate medical education has long been defined by high grades, top test scores, honor society memberships, and publication records. However, this model of learner excellence is misaligned with how students learn or what society values. This accolade-driven view of excellence is perpetuated by assessments that are based on gestalt impressions influenced by similarity between evaluators and students, and assessments that are often restricted to a limited number of traditional skill domains. To achieve a new model of learner excellence that values the trainee's achievement, growth, and responsiveness to feedback across multiple domains, we must envision a new model of teacher excellence. Such teachers would have a growth mindset toward assessing competencies and learning new competencies. Actualizing true learner excellence will require teachers to change from evaluators who conduct assessments of learning to coaches who do assessment for learning. Schools will also need to establish policies and structures that foster a culture that supports this change. In this new paradigm, a teacher's core duty is to develop talent rather than sort it.
Collapse
Affiliation(s)
- Gurpreet Dhaliwal
- Department of Medicine, University of California San Francisco School of Medicine, San Francisco, CA, USA.
- Medical Service, San Francisco VA Medical Center, San Francisco, CA, USA.
| | - Karen E Hauer
- Department of Medicine, University of California San Francisco School of Medicine, San Francisco, CA, USA
| |
Collapse
|
9
|
Richardson D, Kinnear B, Hauer KE, Turner TL, Warm EJ, Hall AK, Ross S, Thoma B, Van Melle E. Growth mindset in competency-based medical education. MEDICAL TEACHER 2021; 43:751-757. [PMID: 34410891 DOI: 10.1080/0142159x.2021.1928036] [Citation(s) in RCA: 54] [Impact Index Per Article: 18.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
The ongoing adoption of competency-based medical education (CBME) across health professions training draws focus to learner-centred educational design and the importance of fostering a growth mindset in learners, teachers, and educational programs. An emerging body of literature addresses the instructional practices and features of learning environments that foster the skills and strategies necessary for trainees to be partners in their own learning and progression to competence and to develop skills for lifelong learning. Aligned with this emerging area is an interest in Dweck's self theory and the concept of the growth mindset. The growth mindset is an implicit belief held by an individual that intelligence and abilities are changeable, rather than fixed and immutable. In this paper, we present an overview of the growth mindset and how it aligns with the goals of CBME. We describe the challenges associated with shifting away from the fixed mindset of most traditional medical education assumptions and practices and discuss potential solutions and strategies at the individual, relational, and systems levels. Finally, we present future directions for research to better understand the growth mindset in the context of CBME.
Collapse
Affiliation(s)
- Denyse Richardson
- Department of Medicine, Division of Physiatry, University of Toronto, Ontario, Canada
- Royal College of Physicians and Surgeons of Canada, Ottawa, Canada
| | - Benjamin Kinnear
- Internal Medicine and Pediatrics, University of Cincinnati College of Medicine, Cincinnati, OH, USA
| | - Karen E Hauer
- University of California, San Francisco, San Francisco, CA, USA
| | - Teri L Turner
- Pediatrics, Baylor College of Medicine, Houston, TX, USA
| | - Eric J Warm
- Internal Medicine and Pediatrics, University of Cincinnati College of Medicine, Cincinnati, OH, USA
| | - Andrew K Hall
- Royal College of Physicians and Surgeons of Canada, Ottawa, Canada
- Department of Emergency Medicine, Queen's University, Kingston, Canada
| | - Shelley Ross
- Department of Family Medicine, University of Alberta, Edmonton, Canada
| | - Brent Thoma
- Royal College of Physicians and Surgeons of Canada, Ottawa, Canada
- Department of Emergency Medicine, University of Saskatchewan, Saskatoon, Canada
| | - Elaine Van Melle
- Royal College of Physicians and Surgeons of Canada, Ottawa, Canada
- Department of Family Medicine, Queen's University, Kingston, Canada
| |
Collapse
|
10
|
Pete Devon E, Burns R, Hartke A. The LETTER of Recommendation: Showcasing a Student's Strengths. Pediatrics 2021; 147:peds.2020-049615. [PMID: 33597284 DOI: 10.1542/peds.2020-049615] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 12/18/2020] [Indexed: 11/24/2022] Open
Affiliation(s)
- Erin Pete Devon
- Department of Pediatrics, Perelman School of Medicine at the University of Pennsylvania, The Children's Hospital of Philadelphia, Philadelphia, Pennsylvania;
| | - Rebekah Burns
- Department of Pediatrics, School of Medicine, University of Washington, Seattle, Washington
| | - Amanda Hartke
- Department of Pediatrics, School of Medicine, University of South Carolina, Columbia, South Carolina; and.,Prisma Health Upstate, Greenville, South Carolina
| |
Collapse
|
11
|
Saudek K, Treat R, Rogers A, Hahn D, Lauck S, Saudek D, Weisgerber M. A novel faculty development tool for writing a letter of recommendation. PLoS One 2020; 15:e0244016. [PMID: 33326489 PMCID: PMC7743943 DOI: 10.1371/journal.pone.0244016] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/23/2020] [Accepted: 11/18/2020] [Indexed: 11/29/2022] Open
Abstract
Objective Based on a national survey of program directors we developed a letter of recommendation (LOR) scoring rubric (SR) to assess LORs submitted to a pediatric residency program. The objective was to use the SR to analyze: the consistency of LOR ratings across raters and LOR components that contributed to impression of the LOR and candidate. Methods We graded 30 LORs submitted to a pediatric residency program that were evenly distributed based on final rank by our program. The SR contained 3 sections (letter features, phrases, and applicant abilities) and 2 questions about the quality of the LOR (LORQ) and impression of the candidate (IC) after reading the LOR on a 5-point Likert scale. Inter-rater reliability was calculated with intraclass correlation coefficients (ICC(2,1)). Pearson (r) correlations and stepwise multivariate linear regression modeling predicted LORQ and IC. Mean scores of phrases, features, and applicant abilities were analyzed with ANOVA and Bonferroni correction. Results Phrases (ICC(2,1) = 0.82, p<0.001)) and features (ICC(2,1) = 0.60, p<0.001)) were rated consistently, while applicant abilities were not (ICC(2,1) = 0.28, p<0.001)). For features, LORQ (R2 = 0.75, p<0.001) and IC (R2 = 0.58, p<0.001) were best predicated by: writing about candidates’ abilities, strength of recommendation, and depth of interaction with the applicant. For abilities, LORQ (R2 = 0.47, p<0.001) and IC (R2 = 0.51, p<0.001) were best predicted by: clinical reasoning, leadership, and communication skills (0.2). There were significant differences for phrases and features (p<0.05). Conclusions The SR was consistent across raters and correlates with impression of LORQ and IC. This rubric has potential as a faculty development tool for writing LORS.
Collapse
Affiliation(s)
- Kris Saudek
- Division of Neonatology, Department of Pediatrics, Medical College of Wisconsin, Milwaukee, Wisconsin, United States of America
- * E-mail:
| | - Robert Treat
- Division of Neonatology, Department of Pediatrics, Medical College of Wisconsin, Milwaukee, Wisconsin, United States of America
| | - Amanda Rogers
- Department of Pediatrics, Division of Hospital Medicine, Milwaukee, Wisconsin, United States of America
| | - Danita Hahn
- Department of Pediatrics, Division of Hospital Medicine, Milwaukee, Wisconsin, United States of America
| | - Sara Lauck
- Department of Pediatrics, Division of Hospital Medicine, Milwaukee, Wisconsin, United States of America
| | - David Saudek
- Department of Pediatrics, Division of Cardiology, Milwaukee, Wisconsin, United States of America
| | - Michael Weisgerber
- Department of Pediatrics, Division of Hospital Medicine, Milwaukee, Wisconsin, United States of America
| |
Collapse
|
12
|
Young M, LaDonna K, Varpio L, Balmer DF. Focal Length Fluidity: Research Questions in Medical Education Research and Scholarship. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2019; 94:S1-S4. [PMID: 31365400 DOI: 10.1097/acm.0000000000002913] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
Research and scholarship in health professions education has been shaped by intended audience (i.e., producers vs users) and the purpose of research questions (i.e., curiosity driven or service oriented), but these archetypal dichotomies do not represent the breadth of scholarship in the field. Akin to an array of lenses required by scientists to capture images of a black hole, the authors propose the analogy of lenses with different focal lengths to consider how different kinds of research questions can offer insight into health professions research-a microscope, a magnifying glass, binoculars, and telescopes allow us to ask and answer different kinds of research questions. They argue for the relevance of all of the different kinds of research questions (or focal lengths); each provides important insight into a particular phenomenon and contributes to understanding that phenomenon in a different way. The authors propose that research questions can move fluidly across focal lengths. For example, a theoretical question can be made more pragmatic through asking "how" questions ("How can we observe and measure a phenomenon?"), whereas a pragmatic question can be made more theoretic by asking a series of "why" questions ("Why are these findings relevant to larger issues?"). In summary, only through the combination of lenses with different focal lengths, brought to bear through interdisciplinary work, can we fully comprehend important phenomena in health professions education and scholarship-the same way scientists managed to image a black hole.
Collapse
Affiliation(s)
- Meredith Young
- M. Young is associate professor, Department of Medicine and Institute for Health Sciences Education, McGill University, Montreal, Quebec, Canada. K. LaDonna is assistant professor, Department of Innovation in Medical Education and Department of Medicine, University of Ottawa, Ottawa, Ontario, Canada. L. Varpio is professor, Department of Medicine, Uniformed Services University of the Health Sciences, Bethesda, Maryland. D.F. Balmer is associate professor, Department of Pediatrics, The Children's Hospital of Pennsylvania and University of Pennsylvania, Philadelphia, Pennsylvania
| | | | | | | |
Collapse
|