A remedy for resident evaluation and remediation

Post graduate remediation programs in medicine: a scoping review

BACKGROUND

Recognizing that physicians may struggle to achieve knowledge, skills, attitudes and or conduct at one or more stages during their training has highlighted the importance of the 'deliberate practice of improving performance through practising beyond one's comfort level under guidance'. However, variations in physician, program, contextual and healthcare and educational systems complicate efforts to create a consistent approach to remediation. Balancing the inevitable disparities in approaches and settings with the need for continuity and effective oversight of the remediation process, as well as the context and population specific nature of remediation, this review will scrutinise the remediation of physicians in training to better guide the design, structuring and oversight of new remediation programs.

METHODS

Krishna's Systematic Evidence Based Approach is adopted to guide this Systematic Scoping Review (SSR in SEBA) to enhance the transparency and reproducibility of this review. A structured search for articles on remediation programs for licenced physicians who have completed their pre-registration postings and who are in training positions published between 1st January 1990 and 31st December 2021 in PubMed, Scopus, ERIC, Google Scholar, PsycINFO, ASSIA, HMIC, DARE and Web of Science databases was carried out. The included articles were concurrently thematically and content analysed using SEBA's Split Approach. Similarities in the identified themes and categories were combined in the Jigsaw Perspective and compared with the tabulated summaries of included articles in the Funnelling Process to create the domains that will guide discussions.

RESULTS

The research team retrieved 5512 abstracts, reviewed 304 full-text articles and included 101 articles. The domains identified were characteristics, indications, frameworks, domains, enablers and barriers and unique features of remediation in licenced physicians in training programs.

CONCLUSION

Building upon our findings and guided by Hauer et al. approach to remediation and Taylor and Hamdy's Multi-theories Model, we proffer a theoretically grounded 7-stage evidence-based remediation framework to enhance understanding of remediation in licenced physicians in training programs. We believe this framework can guide program design and reframe remediation's role as an integral part of training programs and a source of support and professional, academic, research, interprofessional and personal development.

Emotion in remediation: A scoping review of the medical education literature

OBJECTIVES

Remediation can be crucial and high stakes for medical learners, and experts agree it is often not optimally conducted. Research from other fields indicates that explicit incorporation of emotion improves education because of emotion's documented impacts on learning. Because this could present an important opportunity for improving remediation, we aimed to investigate how the literature on remediation interventions in medical education discusses emotion.

METHODS

The authors used Arksey and O'Malley's framework to conduct a scoping literature review of records describing remediation interventions in medical education, using PubMed, CINAHL Complete, ERIC, Web of Science and APA PsycInfo databases, including all English-language publications through 1 May 2020 meeting search criteria. They included publications discussing remediation interventions either empirically or theoretically, pertaining to physicians or physician trainees of any level. Two independent reviewers used a standardised data extraction form to report descriptive information; they reviewed included records for the presence of mentions of emotion, described the mentions and analysed results thematically.

RESULTS

Of 1644 records, 199 met inclusion criteria and were reviewed in full. Of those, 112 (56%) mentioned emotion in some way; others focused solely on cognitive aspects of remediation. The mentions of emotion fell into three themes based on when the emotion was cited as present: during regular coursework or practice, upon referral for remediation and during remediation. One-quarter of records (50) indicated potential intentional incorporation of emotion into remediation programme design, but they were non-specific as to how emotions related to the learning process itself.

CONCLUSION

Even though emotion is omnipresent in remediation, medical educators frequently do not factor emotion into the design of remediation approaches and rarely explicitly utilise emotion to improve the learning process. Applications from other fields may help medical educators leverage emotion to improve learning in remediation, including strategies to frame and design remediation.

Optimising the delivery of remediation programmes for doctors: A realist review

CONTEXT

Medical underperformance puts patient safety at risk. Remediation, the process that seeks to 'remedy' underperformance and return a doctor to safe practice, is therefore a crucially important area of medical education. However, although remediation is used in health care systems globally, there is limited evidence for the particular models or strategies employed. The purpose of this study was to conduct a realist review to ascertain why, how, in what contexts, for whom and to what extent remediation programmes for practising doctors work to restore patient safety.

METHOD

We conducted a realist literature review consistent with RAMESES standards. We developed a programme theory of remediation by carrying out a systematic search of the literature and through regular engagement with a stakeholder group. We searched bibliographic databases (MEDLINE, EMBASE, PsycINFO, HMIC, CINAHL, ERIC, ASSIA and DARE) and conducted purposive supplementary searches. Relevant sections of text relating to the programme theory were extracted and synthesised using a realist logic of analysis to identify context-mechanism-outcome configurations (CMOcs).

RESULTS

A 141 records were included. The majority of the studies were from North America (64%). 29 CMOcs were identified. Remediation programmes are effective when a doctor's insight and motivation are developed and behaviour change reinforced. Insight can be developed by providing safe spaces, using advocacy to promote trust and framing feedback sensitively. Motivation can be enhanced by involving the doctor in remediation planning, correcting causal attribution, goal setting and destigmatising remediation. Sustained change can be achieved by practising new behaviours and skills, and through guided reflection.

CONCLUSION

Remediation can work when it creates environments that trigger behaviour change mechanisms. Our evidence synthesis provides detailed recommendations on tailoring implementation and design strategies to improve remediation interventions for doctors.

Remediation programmes for practising doctors to restore patient safety: the RESTORE realist review

Background

An underperforming doctor puts patient safety at risk. Remediation is an intervention intended to address underperformance and return a doctor to safe practice. Used in health-care systems all over the world, it has clear implications for both patient safety and doctor retention in the workforce. However, there is limited evidence underpinning remediation programmes, particularly a lack of knowledge as to why and how a remedial intervention may work to change a doctor’s practice.

Objectives

To (1) conduct a realist review of the literature to ascertain why, how, in what contexts, for whom and to what extent remediation programmes for practising doctors work to restore patient safety; and (2) provide recommendations on tailoring, implementation and design strategies to improve remediation interventions for doctors.

Design

A realist review of the literature underpinned by the Realist And MEta-narrative Evidence Syntheses: Evolving Standards quality and reporting standards.

Data sources

Searches of bibliographic databases were conducted in June 2018 using the following databases: EMBASE, MEDLINE, Cumulative Index to Nursing and Allied Health Literature, PsycINFO, Education Resources Information Center, Database of Abstracts of Reviews of Effects, Applied Social Sciences Index and Abstracts, and Health Management Information Consortium. Grey literature searches were conducted in June 2019 using the following: Google Scholar (Google Inc., Mountain View, CA, USA), OpenGrey, NHS England, North Grey Literature Collection, National Institute for Health and Care Excellence Evidence, Electronic Theses Online Service, Health Systems Evidence and Turning Research into Practice. Further relevant studies were identified via backward citation searching, searching the libraries of the core research team and through a stakeholder group.

Review methods

Realist review is a theory-orientated and explanatory approach to the synthesis of evidence that seeks to develop programme theories about how an intervention produces its effects. We developed a programme theory of remediation by convening a stakeholder group and undertaking a systematic search of the literature. We included all studies in the English language on the remediation of practising doctors, all study designs, all health-care settings and all outcome measures. We extracted relevant sections of text relating to the programme theory. Extracted data were then synthesised using a realist logic of analysis to identify context–mechanism–outcome configurations.

Results

A total of 141 records were included. Of the 141 studies included in the review, 64% related to North America and 14% were from the UK. The majority of studies (72%) were published between 2008 and 2018. A total of 33% of articles were commentaries, 30% were research papers, 25% were case studies and 12% were other types of articles. Among the research papers, 64% were quantitative, 19% were literature reviews, 14% were qualitative and 3% were mixed methods. A total of 40% of the articles were about junior doctors/residents, 31% were about practicing physicians, 17% were about a mixture of both (with some including medical students) and 12% were not applicable. A total of 40% of studies focused on remediating all areas of clinical practice, including medical knowledge, clinical skills and professionalism. A total of 27% of studies focused on professionalism only, 19% focused on knowledge and/or clinical skills and 14% did not specify. A total of 32% of studies described a remediation intervention, 16% outlined strategies for designing remediation programmes, 11% outlined remediation models and 41% were not applicable. Twenty-nine context–mechanism–outcome configurations were identified. Remediation programmes work when they develop doctors’ insight and motivation, and reinforce behaviour change. Strategies such as providing safe spaces, using advocacy to develop trust in the remediation process and carefully framing feedback create contexts in which psychological safety and professional dissonance lead to the development of insight. Involving the remediating doctor in remediation planning can provide a perceived sense of control in the process and this, alongside correcting causal attribution, goal-setting, destigmatising remediation and clarity of consequences, helps motivate doctors to change. Sustained change may be facilitated by practising new behaviours and skills and through guided reflection.

Limitations

Limitations were the low quality of included literature and limited number of UK-based studies.

Future work

Future work should use the recommendations to optimise the delivery of existing remediation programmes for doctors in the NHS.

Study registration

This study is registered as PROSPERO CRD42018088779.

Funding

This project was funded by the National Institute for Health Research (NIHR) Health Services and Delivery Research programme and will be published in full in Health Services and Delivery Research; Vol. 9, No. 11. See the NIHR Journals Library website for further project information.

Principles of Remediation for the Struggling Neurosurgery Resident

BACKGROUND

When resident physicians fail to demonstrate appropriate milestone competencies early in their neurologic surgery residency, a plan of remediation is necessary.

METHODS

Once any psychologic, physical, or behavioral causes of identified knowledge or psychomotor deficiencies have been identified and addressed, the next step is to develop a plan to close these gaps. Specific areas that are assessed for deficits include medical knowledge, clinical reasoning and judgment, clinical skills, time management and organization, interpersonal skills, communication, and professionalism. Specific learning goals and objectives, as well as teaching and learning methods, pertain to the unique areas of deficit, and all of these must be considered with the goal of developing a resident-specific remediation plan.

RESULTS

A plan for assessment of the remediation process is described, including an evaluation of what constitutes individual resident remediation success.

CONCLUSIONS

Finally, a discussion of the prior resident remediation studies across many disciplines is made.

Remediation of learners struggling with communication skills: a systematic review

BACKGROUND

Communication skills is a core area of competency for healthcare practitioners. However, trainees deficient in those skills are not identified early enough to address the deficiency. Furthermore, faculty often struggle to identify effective remediation strategies for those who fail to meet expectations. We undertook a systematic review to determine which assessment methods are appropriate to identify learners that struggle with communication skills and the strategies used to remediate them.

METHODS

The literature was searched from January 1998 through to May 2019 using academic databases and grey literature. Trainees were defined as healthcare practitioners in undergraduate, graduate and continuing education. Characteristics of studies, assessment and intervention strategies and outcomes were synthesized qualitatively and summarized in tables.

RESULTS

From an initial 1636 records, 16 (1%) studies met the review criteria. Majority of the learners were medical students. A few studies (44%) included students from other disciplines, residents and physicians in practice. The remediation programs, in the studies, ranged from 1 week to 1 year. Around half of the studies focused solely on learners struggling with communication skills. The majority of studies used a format of a clinical OSCE to identify struggling learners. None of the studies had a single intervention strategy with the majority including an experiential component with feedback.

CONCLUSIONS

A few studies collectively described the diagnosis, remediation intervention and the assessment of the outcomes of remediation of communication skills. For a remediation strategy to be successful it is important to ensure: (i) early identification and diagnosis, (ii) the development of an individualized plan and (iii) providing reassessment with feedback to the learner.

A qualitative study of medical educators' perspectives on remediation: Adopting a holistic approach to struggling residents

INTRODUCTION

During residency, some trainees require the identification and remediation of deficiencies to achieve the knowledge, skills and attitudes necessary for independent practice. Given the limited published frameworks for remediation, we characterize remediation from the perspective of educators and propose a holistic framework to guide the approach to remediation.

METHODS

We conducted semistructured focus groups to: explore methods for identifying struggling residents; categorize common domains of struggle; describe personal factors that contribute to difficulties; define remediation interventions and understand what constitutes successful completion. Data were analyzed through conventional content analysis.

RESULTS

Nineteen physicians across multiple specialties and institutions participated in seven focus groups. Thirteen categories emerged around remediation. Some themes addressed practical components of remediation, while others reflected barriers to the process and the impact of remediation on the resident and program. The themes were used to inform development of a novel holistic framework for remediation.

CONCLUSIONS

The approach to remediation requires comprehensive identification of individual factors impacting performance. The intervention should not only include a tailored learning plan but also address confounders that impact likelihood of remediation success. Our holistic framework intends to guide educators creating remediation plans to ensure all domains are addressed.

The threat of the dyscompetent resident: A plea to make the implicit more explicit!

Although several examples of frameworks dealing with students' unprofessional behaviour are available, guidance on how to deal locally or regionally with dysfunctional residents is limited (Hickson et al. in Acad Med 82(11):1040-1048, 2007b; Leape and Fromson in Ann Intern Med 144(2):107-115, 2006). Any 'rules' are mostly unwritten, and often emerge by trial and error within the specialty training programme (Stern and Papadakis in N Engl J Med 355(17):1794-1799, 2006). It is nevertheless of utmost importance that objectives, rules and guidelines comparable to those existing in undergraduate training (Project Team Consilium Abeundi van Luijk in Professional behaviour: teaching, assessing and coaching students. Final report and appendices. Mosae Libris, 2005; van Mook et al. in Neth J Crit Care 16(4):162-173, 2010a) are developed for postgraduate training. And that implicit rules are made explicit. This article outlines a framework based on the lessons learned from contemporary postgraduate medical training programmes.

Remedial training for the radiology resident: a template for optimization of the learning plan

All radiology residency programs should strive for the early identification of individuals in need of remedial training and have an approach ready to address this situation. This article provides a template for a step-by-step approach which is team based. It includes definition of the learning or performance issues, creation of suitable learning objectives and learning plan, facilitation of feedback and assessment, and definition of outcomes. Using such a template will assist the resident in returning to the path toward a safe and competent radiologist.

Remediation of residents in difficulty: a retrospective 10-year review of the experience of a postgraduate board of examiners

PURPOSE

To determine, through a 10-year review, (1) the prevalence of residents in difficulty, (2) characteristics of these residents, (3) areas of residents' weakness, and (4) outcomes of residents who undergo remediation.

METHOD

A retrospective review of resident records for the University of Toronto Faculty of Medicine's (UT-FOM) Board of Examiners for Postgraduate Programs (BOE-PG) was done from July 1, 1999 to June 30, 2009 using predetermined data elements entered into a standardized form and analyzed for trends and significance. Outcomes for residents in difficulty were tracked through university registration systems and licensure databases.

RESULTS

During 10 years, 103 UT-FOM residents were referred to the BOE-PG, representing 3% of all residents enrolled. The annual prevalence of residents referred to the BOE-PG ranged from 0.2% to 1.5%. The CanMEDS framework was used to classify areas of residents' weaknesses and organize remediation plans. All 100 residents studied had either medical expertise (85%) or professionalism (15%) weaknesses or both. Residents had difficulties with an average of 2.6 CanMEDS Roles, with highest frequencies of Medical Expert (85%) Professional (51%), Communicator (49%), Manager (43%), and Collaborator (20%). Often, there were multiple remediation periods, with an average of six months' duration. Usually, remediation was successful; 78% completed residency education, 17% were unsuccessful, and 5% remained in training.

CONCLUSION

Residents in difficulty have multiple areas of weakness. The CanMEDS framework is an effective approach to classifying problems and designing remediation plans. Successful completion of residency education after remediation is the most common outcome.

Determining need for remediation through postrotation evaluations

INTRODUCTION

Postrotation evaluations are frequently used by residency program directors for early detection of residents with academic difficulties; however, the accuracy of these evaluations in assessing resident performance has been questioned.

METHODS

This retrospective case-control study examines the ability of postrotation evaluation characteristics to predict the need for remediation. We compared the evaluations of 17 residents who were placed on academic warning or probation, from 2000 to 2007, with those for a group of peers matched on sex, postgraduate year (PGY), and entering class.

RESULTS

The presence of an outlier evaluation, the number of words written in the comments section, and the percentage of evaluations with negative or ambiguous comments were all associated with the need for remediation (P  =  .01, P  =  .001, P  =  .002, P  =  < .001, respectively). In contrast, United States Medical Licensing Examination step 1 and step 2 scores, total number of evaluations received, and percentage of positive comments on the evaluations were not associated with the need for remediation (P  =  .06, P  =  .87, P  =  .55, respectively).

DISCUSSION

Despite ambiguous evaluation comments, the length and percentage of ambiguous or negative comments did indicate future need for remediation.

CONCLUSIONS

Our study demonstrates that postrotation evaluation characteristics can be used to identify residents as risk. However, larger prospective studies, encompassing multiple institutions, are needed to validate various evaluation methods in measuring resident performance and to accurately predict the need for remediation.

Resident evaluation and remediation: a comprehensive approach

BACKGROUND

A comprehensive evaluation and remediation program is an essential component of any residency program. The evaluation system should identify problems accurately and early and allow residents with problems to be assigned to a remediation program that effectively deals with them. Elements of a proactive remediation program include a process for outlining deficiencies, providing resources for improvement, communicating clear goals for acceptable performance, and reevaluating performance against these goals.

INTERVENTION

In recognition of the importance of early detection and prompt remediation of the struggling resident, we sought to develop a multifaceted approach to resident evaluation with the aim of early identification and prompt remediation of difficulties. This article describes our comprehensive evaluation program and remediation program, which uses resources within our radiology department and institutional graduate medical education office.

DISCUSSION

An effective evaluation system should identify problems accurately and early, whereas a proactive remediation program should effectively deal with issues once they are identified.

Remediation practices for surgery residents

BACKGROUND

This study sought to determine to what extent surgery programs are remediating residents who fail to achieve competency and to offer remediation strategies.

METHODS

A web-based survey was e-mailed to 253 program directors of all US surgery residency programs. Questions were asked about remediation and probation practices for residents failing to meet the competencies.

RESULTS

Programs seem to struggle the least with knowing how to remediate medical knowledge and patient care deficits and struggle more with professionalism and interpersonal communication skills. Most programs have no remediation methods in place for systems-based practice and practice-based learning and improvement deficits.

CONCLUSIONS

Surgery residency programs are cognizant of the reality that some residents perform unsatisfactorily. Most have remediation plans for residents and understand that a process needs to be in place. Remediation methods tend to vary depending on the deficit and are devised tailored to the resident's needs.

A stitch in time saves nine: intervention strategies for the remediation of competency

The mission of medical educators in all fields is to perfect the science and the art of training competent physicians and the next generation of leaders. To do this, outcomes research regarding effective training and supervision techniques and curricula, the optimal educational environment, assessment, and remediation of competency is needed. It is incumbent upon training directors to continue to provide the primary leadership for the field of training. The areas that are ripe for research and development include effective curricula, effective evaluation and remediation of competency, and leadership skill development. As medicine becomes more complex, so does the task of training physicians. Joint monitoring and outcomes research of the training enterprise are the scholarly and academic missions of the present and the future.

Impact of Electronic Resident Evaluation Systems on the Timeliness, Rate, and Quality of Responses

OBJECTIVE

To determine the impact of electronic resident evaluation systems on the timeliness, rate, and quality of responses.

METHODS

Surveys were mailed electronically to the membership of the Association of Program Directors in Radiology, which included the directors of 158 residency programs. Respondents were instructed to send 1 response from each program. Information gathered included the use of electronic compared with paper-based evaluation systems and the overall level of satisfaction with such systems (rated on a 5-point scale with 1 = dissatisfied and 5 = satisfied). Questions were asked regarding the impact of electronic systems on the timeliness, rate, and quality of responses.

RESULTS

Seventy-seven responses were received, for an estimated response rate of 49% on a per-program basis. Of these 77 respondents, 45 (58%) used electronic systems and 32 (42%) used paper-based systems. The median level of satisfaction was equivalent (4 = somewhat satisfied) for both groups. Of the 45 respondents using electronic systems, 26 (58%) reported increased response rates, 16 (36%) reported no change, and 3 (7%) reported decreased response rates compared with paper-based systems. Regarding the timeliness of responses, 31 (69%) reported faster response rates, 10 (22%) reported no change, and 4 (9%) reported slower rates compared with paper-based systems. Regarding the quality of responses, 25 (56%) reported no change, 12 (27%) reported improved quality, and 8 (18%) reported worse quality compared with paper-based systems.

CONCLUSION

Electronic systems are generally associated with an improved response rate and enhanced timeliness of responses in comparison with paper-based systems, without adversely affecting the quality of responses.