Clinical teaching performance improvement of faculty in residency training: A prospective cohort study

Practice makes perfect: The learner-doctor blueprint for proficient physicians

In the process of producing competent and efficient medical graduates, clinical teaching is an integral component of undergraduate training. Even though clinical training has been linked with multiple benefits and plays a defining role in transforming a medical student into a competent doctor, it has some inherent challenges. The learner-doctor method emphasizes learning by doing and has the potential to counter the identified challenges in the traditional mode of clinical training. The learner-doctor method places emphasis on collaborative learning, wherein students learn by experiential learning and active participation in patient care. This method not only fosters the development of clinical skills but also cultivates critical thinking, teamwork, empathy, and effective communication. A number of activities or strategies must be planned and implemented to ensure that the set objectives of the learner-doctor method are accomplished by the medical institutions. In conclusion, as medical education continues to evolve, the learner-doctor method of clinical training is a dynamic approach to bridging the gap between theoretical knowledge and practical skills. As this method enables medical students to actively engage in patient care and improve their critical thinking, it becomes the need of the hour that every medical institution should explore the opportunity to effectively implement the same in their settings.

Linking leadership development programs for physicians with organization-level outcomes: a realist review

BACKGROUND

Hospitals invest in Leadership Development Programs (LDPs) for physicians, assuming they benefit the organization's performance. Researchers have listed the advantages of LDPs, but knowledge of how and why organization-level outcomes are achieved is missing.

OBJECTIVE

To investigate how, why and under which circumstances LDPs for physicians can impact organization-level outcomes.

METHODS

We conducted a realist review, following the RAMESES guidelines. Scientific articles and grey literature published between January 2010 and March 2021 evaluating a leadership intervention for physicians in the hospital setting were considered for inclusion. The following databases were searched: Medline, PsycInfo, ERIC, Web of Science, and Academic Search Premier. Based on the included documents, we developed a LDP middle-range program theory (MRPT) consisting of Context-Mechanism-Outcome configurations (CMOs) describing how specific contexts (C) trigger certain mechanisms (M) to generate organization-level outcomes (O).

RESULTS

In total, 3904 titles and abstracts and, subsequently, 100 full-text documents were inspected; 38 documents with LDPs from multiple countries informed our MRPT. The MRPT includes five CMOs that describe how LDPs can impact the organization-level outcomes categories 'culture', 'quality improvement', and 'the leadership pipeline': 'Acquiring self-insight and people skills (CMO1)', 'Intentionally building professional networks (CMO2)', 'Supporting quality improvement projects (CMO3)', 'Tailored LDP content prepares physicians (CMO4)', and 'Valuing physician leaders and organizational commitment (CMO5)'. Culture was the outcome of CMO1 and CMO2, quality improvement of CMO2 and CMO3, and the leadership pipeline of CMO2, CMO4, and CMO5. These CMOs operated within an overarching context, the leadership ecosystem, that determined realizing and sustaining organization-level outcomes.

CONCLUSIONS

LDPs benefit organization-level outcomes through multiple mechanisms. Creating the contexts to trigger these mechanisms depends on the resources invested in LDPs and adequately supporting physicians. LDP providers can use the presented MRPT to guide the development of LDPs when aiming for specific organization-level outcomes.

Only When They Seek: Exploring Supervisor and Resident Perspectives and Positions on Upward Feedback

INTRODUCTION

Verbal feedback from trainees to supervisors is rare in medical education, although valuable for improvement in teaching skills. Research has mostly examined narrative comments on resident evaluations of their supervisors. This study aimed to explore supervisors' and residents' beliefs and experiences with upward feedback, along with recommendations to initiate and facilitate effective conversations.

METHODS

Using 60-minute focus group discussions, a previous study explored opinions of internal medicine residents and clinical supervisors at the Brigham and Women's Hospital regarding the impact of institutional culture on feedback conversations. For this study, we conducted a thematic analysis of the transcribed, anonymous data to identify key concepts pertaining only to verbal upward feedback, through the theoretical lens of Positioning theory.

RESULTS

Twenty-two supervisors and 29 residents participated in three and five focus groups, respectively. Identified themes were mapped to three research questions regarding (1) existing beliefs (lack of impact, risks to giving supervisors feedback, need for preparation and reflection), (2) experiences (nonspecific language, avoidance of upward feedback, bypassing the supervisor), and (3) recommended approaches (setting clear expectations, seeking specific feedback, emphasizing interest in growth).

DISCUSSION

Study participants appeared to assume learner-teacher positions during feedback conversations, resulting in residents' concerns of adverse consequences, beliefs that supervisors will neither accept feedback nor change their behaviors, and avoidance of constructive upward feedback. Residents suggested that emphasis on mutual professional growth and regular feedback seeking by supervisors could encourage them to take on the role of feedback providers. Their recommendations could be a valuable starting point for faculty development initiatives on upward feedback.

Improving Faculty Teaching Practices: Fostering the "Ideal ICU Teacher" Using a Positive Deviance Strategy

We describe an innovative approach to medical education, using "positive deviants," or faculty who emerge as role models, despite responsibilities that compete with teaching efforts. In small group sessions, "positive deviants" taught effective strategies, resulting in faculty implementing new strategies.

Rethinking Our Annual Congress-Meeting the Needs of Specialist Physicians by Partnering With Provincial Simulation Centers

Canada's maintenance of certification programs for physicians has evolved to emphasize assessment activities. Our organization recognized the importance of offering more practice assessment opportunities to our members to enhance their practice and help them comply with a regulation from our provincial professional body related to ongoing continuing education. This led us to rethink our annual congress and enrich the program with a curriculum of interdisciplinary simulation sessions tailored to meet the needs of a broad audience of specialists. Our challenges are similar to those of many national specialty societies having limited access to simulation facilities, instructors, and simulation teams that can cover the breadth and scope of perceived and unperceived simulation needs for their specialty. Our innovative solution was to partner with local experts to develop 22 simulation sessions over the past three years. The response was very positive, drawing 867 participants. Over 95% of participants either agreed or strongly agreed that their simulation session (1) met their learning objectives, (2) was relevant for their practice, and (3) encouraged them to modify their practice. Narrative comments from a survey sent to the 2018 participants four months after their activity indicated several self-reported changes in their practice or patient outcomes. We were able to centralize offers from organizations that had previously worked in silo to develop simulation sessions meeting the needs of our members. Proposing simulation sessions allowed our organization to establish long-term partnerships and to expend our "educational toolbox" to address skill gaps not usually addressed during annual meetings.

Teaching quality in neurosurgery: quantitating outcomes over time

OBJECTIVE

High-quality neurosurgery resident training is essential to developing competent neurosurgeons. Validated formative tools to assess faculty teaching performance exist, but are not used widely among Accreditation Council for Graduate Medical Education (ACGME) residency programs in the United States. Furthermore, their longer-term impact on teaching performance improvement and educational outcomes remains unclear. The goal of this study was to assess the impact of implementing an evaluation system to provide faculty with feedback on teaching performance in a neurosurgery residency training program over a 4-year period.

METHODS

The authors performed a prospective cohort study in which a modified version of the System for Evaluation of Teaching Qualities (SETQ) instrument was administered to neurosurgical trainees in their department regularly every 6 months. The authors analyzed subscale score dynamics to identify the strongest correlates of faculty teaching performance improvement. ACGME program survey results and trainee performance on written board examinations were compared for the 3 years before and after SETQ implementation.

RESULTS

The overall response rate among trainees was 91.8%, with 1044 surveys completed for 41 faculty. Performance scores improved progressively from cycle 1 to cycle 6. The strongest correlate of overall performance was providing positive feedback to trainees. Compared to the 3 years prior, the 3 years following SETQ implementation saw significant increases in written board examination and ACGME resident survey scores compared to the national mean.

CONCLUSIONS

Implementation of SETQ was associated with significant improvements in faculty teaching performance as judged by trainees over a 4-year period, and guided curricular changes in the authors' training program that resulted in improved educational outcomes.

An Examination of Self-Reported Assessment Activities Documented by Specialist Physicians for Maintenance of Certification

INTRODUCTION

Specialists in a Maintenance of Certification program are required to participate in assessment activities, such as chart audit, simulation, knowledge assessment, and multisource feedback. This study examined data from five different specialties to identify variation in participation in assessment activities, examine differences in the learning stimulated by assessment, assess the frequency and type of planned changes, and assess the association between learning, discussion, and planned changes.

METHODS

E-portfolio data were categorized and analyzed descriptively. Chi-squared tests examined associations.

RESULTS

A total of 2854 anatomical pathologists, cardiologists, gastroenterologists, ophthalmologists, and orthopedic surgeons provided data about 6063 assessment activities. Although there were differences in the role that learning played by discipline and assessment type, the most common activities documented across all specialties were self-assessment programs (n = 2122), feedback on teaching (n = 1078), personal practice assessments which the physician did themselves (n = 751), annual reviews (n = 682), and reviews by third parties (n = 661). Learning occurred for 93% of the activities and was associated with change. For 2126 activities, there were planned changes. Activities in which there was a discussion with a peer or supervisor were more likely to result in a change.

CONCLUSIONS AND DISCUSSION

Although specialists engaged in many types of assessment activities to meet the Maintenance of Certification program requirements, there was variability in how assessment stimulated learning and planned changes. It seems that peer discussion may be an important component in fostering practice change and forming plans for improvement which bears further study.

Exploring Validity Evidence Associated With Questionnaire-Based Tools for Assessing the Professional Performance of Physicians: A Systematic Review

PURPOSE

To collect and examine-using an argument-based validity approach-validity evidence of questionnaire-based tools used to assess physicians' clinical, teaching, and research performance.

METHOD

In October 2016, the authors conducted a systematic search of the literature seeking articles about questionnaire-based tools for assessing physicians' professional performance published from inception to October 2016. They included studies reporting on the validity evidence of tools used to assess physicians' clinical, teaching, and research performance. Using Kane's validity framework, they conducted data extraction based on four inferences in the validity argument: scoring, generalization, extrapolation, and implications.

RESULTS

They included 46 articles on 15 tools assessing clinical performance and 72 articles on 38 tools assessing teaching performance. They found no studies on research performance tools. Only 12 of the tools (23%) gathered evidence on all four components of Kane's validity argument. Validity evidence focused mostly on generalization and extrapolation inferences. Scoring evidence showed mixed results. Evidence on implications was generally missing.

CONCLUSIONS

Based on the argument-based approach to validity, not all questionnaire-based tools seem to support their intended use. Evidence concerning implications of questionnaire-based tools is mostly lacking, thus weakening the argument to use these tools for formative and, especially, for summative assessments of physicians' clinical and teaching performance. More research on implications is needed to strengthen the argument and to provide support for decisions based on these tools, particularly for high-stakes, summative decisions. To meaningfully assess academic physicians in their tripartite role as doctor, teacher, and researcher, additional assessment tools are needed.

The Association Between Residency Learning Climate and Inpatient Care Experience in Clinical Teaching Departments in the Netherlands

PURPOSE

To examine the association between residency learning climate and inpatient care experience.

METHOD

The authors analyzed 1,201 evaluations of the residency learning climate (using the Dutch Residency Educational Climate Test questionnaire) and 6,689 evaluations of inpatient care experience (using the Consumer Quality Index Inpatient Hospital Care questionnaire) from 86 departments across 15 specialties in 18 hospitals in the Netherlands between 2013 and 2014. The authors used linear hierarchical panel analyses to study the associations between departments' overall and subscale learning climate scores and inpatient care experience global ratings and subscale scores, controlling for respondent- and department-level characteristics and correcting for multiple testing.

RESULTS

Overall learning climate was not associated with global department ratings (b = 0.03; 95% confidence interval -0.17 to 0.23) but was positively associated with specific inpatient care experience domains, including communication with doctors (b = 0.11; 0.02 to 0.20) and feeling of safety (b = 0.09; 0.01 to 0.17). Coaching and assessment was positively associated with communication with doctors (b = 0.22; 0.08 to 0.37) and explanation of treatment (b = 0.22; 0.08 to 0.36). Formal education was negatively associated with pain management (b = -0.16; -0.26 to -0.05), while peer collaboration was positively associated with pain management (b = 0.14; 0.03 to 0.24).

CONCLUSIONS

Optimizing the clinical learning environment is an important step toward ensuring high-quality residency training and patient care. These findings could help clinical teaching departments address those aspects of the learning environment that directly affect patient care.

Perioperative Teaching and Feedback: How are we doing in Canadian OTL-HNS programs?

Background

Discrepancies between resident and faculty perceptions regarding optimal teaching and feedback during surgery are well known but these differences have not yet been described in Otolaryngology - Head and Neck Surgery (OTL-HNS). The objectives were thus to compare faculty and resident perceptions of perioperative teaching and feedback in OTL-HNS residency programs across Canada with the aim of highlighting potential areas for improvement.

Methods

An anonymous electronic questionnaire was distributed to residents and teaching faculty in OTL-HNS across Canada with additional paper copies distributed at four institutions. Surveys consisted of ratings on a 5-point Likert scale and open-ended questions. Responses among groups were analysed with the Wilcoxon-Mann Whitney test, while thematic analysis was used for the open-ended questions.

Results

A total of 143 teaching faculty and residents responded with statistically significant differences on 11 out of 25 variables. Namely, faculty reported higher rates of pre and intra-operative teaching compared to resident reports. Faculty also felt they gave adequate feedback on residents’ strengths and technical skills contrary to what the residents thought. Both groups did agree however that pre-operative discussion is not consistently done, nor is feedback consistently given or sought.

Conclusion

Faculty and residents in OTL-HNS residency programs disagree on the frequency and optimal timing of peri-operative teaching and feedback. This difference in perception emphasizes the need for a more structured approach to feedback delivery including explicitly stating when feedback is being given, and the overall need for better communication between residents and staff.

Surgeons have an opportunity to improve teaching quality through feedback provision

BACKGROUND

Medical student evaluations of faculty are increasingly incorporated into promotion and tenure decisions, making it imperative to understand learner perceptions of quality teaching. Prior work has shown that students value faculty responsiveness in the form of feedback, but faculty and students differ in their perceptions of what constitutes sufficient feedback. The innovative minute feedback system (MFS) can quantify responsiveness to students' feedback requests. This study assessed how feedback provision via MFS impacts teaching quality scores.

MATERIALS AND METHODS

This retrospective observational study compared average faculty teaching quality scores with faculty's percentage response to student feedback requests via the MFS. The data were generated from the core surgical clerkship for third-year medical students at the University of Michigan Medical School. The relationship between average teaching quality scores and response percentage was assessed by weighted regression analysis.

RESULTS

Two hundred thirty-seven medical students requested feedback via MFS, and 104 faculty were evaluated on teaching quality. The mean faculty feedback response percentage was 55.78%. The mean teaching quality score was 4.27 on a scale of 1 to 5. Teaching quality score was significantly correlated with response percentage (P < 0.001); for every 10% increase in response percentage, average teaching quality score improved by 0.075. Average teaching quality score was not significantly associated with response time (P = 0.158), gender (P = 0.407), or surgical service (P = 0.498).

CONCLUSIONS

Medical students consider responsiveness to feedback requests an important component of quality teaching. Furthermore, faculty development focused on efficient and practical feedback strategies may have the added benefit of improving their teaching quality.

Redesign of the System for Evaluation of Teaching Qualities in Anesthesiology Residency Training (SETQ Smart)

BACKGROUND

Given the increasing international recognition of clinical teaching as a competency and regulation of residency training, evaluation of anesthesiology faculty teaching is needed. The System for Evaluating Teaching Qualities (SETQ) Smart questionnaires were developed for assessing teaching performance of faculty in residency training programs in different countries. This study investigated (1) the structure, (2) the psychometric qualities of the new tools, and (3) the number of residents' evaluations needed per anesthesiology faculty to use the instruments reliably.

METHODS

Two SETQ Smart questionnaires-for faculty self-evaluation and for resident evaluation of faculty-were developed. A multicenter survey was conducted among 399 anesthesiology faculty and 430 residents in six countries. Statistical analyses included exploratory factor analysis, reliability analysis using Cronbach α, item-total scale correlations, interscale correlations, comparison of composite scales to global ratings, and generalizability analysis to assess residents' evaluations needed per faculty.

RESULTS

In total, 240 residents completed 1,622 evaluations of 247 faculty. The SETQ Smart questionnaires revealed six teaching qualities consisting of 25 items. Cronbach α's were very high (greater than 0.95) for the overall SETQ Smart questionnaires and high (greater than 0.80) for the separate teaching qualities. Interscale correlations were all within the acceptable range of moderate correlation. Overall, questionnaire and scale scores correlated moderately to highly with the global ratings. For reliable feedback to individual faculty, three to five resident evaluations are needed.

CONCLUSIONS

The first internationally piloted questionnaires for evaluating individual anesthesiology faculty teaching performance can be reliably, validly, and feasibly used for formative purposes in residency training.

Validation of the INCEPT: A Multisource Feedback Tool for Capturing Different Perspectives on Physicians' Professional Performance

INTRODUCTION

Multisource feedback (MSF) instruments are used to and must feasibly provide reliable and valid data on physicians' performance from multiple perspectives. The "INviting Co-workers to Evaluate Physicians Tool" (INCEPT) is a multisource feedback instrument used to evaluate physicians' professional performance as perceived by peers, residents, and coworkers. In this study, we report on the validity, reliability, and feasibility of the INCEPT.

METHODS

The performance of 218 physicians was assessed by 597 peers, 344 residents, and 822 coworkers. Using explorative and confirmatory factor analyses, multilevel regression analyses between narrative and numerical feedback, item-total correlations, interscale correlations, Cronbach's α and generalizability analyses, the psychometric qualities, and feasibility of the INCEPT were investigated.

RESULTS

For all respondent groups, three factors were identified, although constructed slightly different: "professional attitude," "patient-centeredness," and "organization and (self)-management." Internal consistency was high for all constructs (Cronbach's α ≥ 0.84 and item-total correlations ≥ 0.52). Confirmatory factor analyses indicated acceptable to good fit. Further validity evidence was given by the associations between narrative and numerical feedback. For reliable total INCEPT scores, three peer, two resident and three coworker evaluations were needed; for subscale scores, evaluations of three peers, three residents and three to four coworkers were sufficient.

DISCUSSION

The INCEPT instrument provides physicians performance feedback in a valid and reliable way. The number of evaluations to establish reliable scores is achievable in a regular clinical department. When interpreting feedback, physicians should consider that respondent groups' perceptions differ as indicated by the different item clustering per performance factor.