Remediation practices for surgery residents

Performance in Irish Selection and Future Performance in Surgical Training

Importance

Selection processes for surgical training should aim to identify candidates who will become competent independent practitioners and should aspire to high standards of reliability and validity.

Objective

To determine the association between measured candidate factors at the time of an Irish selection and assessment outcomes in surgical training, examined via rate of progression to Higher Specialist Training (HST), attrition rates, and performance as assessed through a multimodal framework of workplace-based and simulation-based assessments.

Design, Setting, and Participants

This retrospective observational cohort study included data from all successful applicants to the Royal College of Surgeons in Ireland (RCSI) national Core Surgical Training (CST) program. Participants included all trainees recruited to dedicated postgraduate surgical training from 2016 to 2020. These data were analyzed from July 11, 2016, through July 10, 2022.

Exposures

Selection decisions were based on a composite score that was derived from technical aptitude assessments, undergraduate academic performance, and a 4-station multiple mini-interview.

Main outcomes and measures

Assessment data, attrition rates, and rates of progression to HST were recorded for each trainee. CST performance was assessed using workplace-based and simulation-based technical and nontechnical skill assessments. Potential associations between selection and assessment measures were explored using Pearson correlation, logistic regression, and multiple linear-regression analyses.

Results

Data were available for 303 trainees. Composite scores were positively associated with progression to HST (odds ratio [OR], 1.09; 95% CI, 1.05-1.13). There was a weak positive correlation, ranging from 0.23 to 0.34, between scores and performance across all CST assessments. Multivariable linear regression analysis showed technical aptitude scores at application were associated with future operative performance assessment scores, both in the workplace (β = 0.31; 95% CI, 0.14-0.48) and simulated environments (β = 0.57; 95% CI, 0.33-0.81). There was evidence that the interpersonal skills interview station was associated with future performance in simulated communication skill assessments (β = 0.55; 95% CI, 0.22-0.87).

Conclusions and Relevance

In this study, performance at the time of Irish national selection, measured across technical and nontechnical domains in a multimodal fashion, was associated with future performance in the workplace and in simulated environments. Future studies will be required to explore the consequential validity of selection, including potential unintended effects of selection and ranking on candidate performance.

Dutch dismissal practices: characteristics, consequences, and contrasts in residents' case law in community-based practice versus hospital-based specialties

BACKGROUND

In the Netherlands, 2 to 10% of the residents terminate training prematurely. Infrequently, termination of training is by dismissal. Incidentally, residents may disagree, dispute and challenge these decisions from the programme directors. Resident dismissal is always a difficult decision, most commonly made after, repeated assessments, and triangulation of the resulting assessment data and one or more remediation attempts. Nevertheless, the underlying reasons for dismissal and the policies for remediation and dismissal may differ between training programmes. Such differences may however impact the chance of remediation success, the chance of dismissal and subsequent residents' appeals.

METHOD

We included a total of 70 residents from two groups (community-based and hospital-based specialties) during 10 years of appeals. Subsequently, we compared these groups on factors potentially associated with the outcome of the conciliation board decision regarding the residents' dismissal. We focused herein on remediation strategies applied, and reasons reported to dismiss residents.

RESULTS

In both groups, the most alleged reason to dismiss residents was lack of trainability, > 97%. This was related to deficiencies in professionalism in community-based practice and medical expertise in hospital-based specialties respectively. A reason less frequently mentioned was endangerment of patient care, < 26%. However, none of these residents accused of endangerment, actually jeopardized the patients' health, probably due to the vigilance of their supervisors. Remediation strategies varied between the two groups, whereas hospital-based specialties preferred formal remediation plans in contrast to community-based practice. A multitude of remediation strategies per competency (medical expertise, professionalism, communication, management) were applied and described in these law cases.

DISCUSSION

Residents' appeals in community-based practice were significantly less likely to succeed compared to hospital-based specialties. Hypothesised explanatory factors underlying these differences include community-based practices' more prominent attention to the longitudinal assessment of professionalism, the presence of regular quarterly progress meetings, precise documentation of deficiencies, and discretion over the timing of dismissal in contrast to dismissal in the hospital-based specialties which is only formally possible during scheduled formal summative assessment meetings.

Identifying Strategies for Struggling Surgery Residents

BACKGROUND

Struggling residents are not uncommon in general surgery. Early identification of these residents and effective remediation remain imperfect.

MATERIALS AND METHODS

We performed a survey of program directors (PD) across all general surgery residencies. Survey questions included the following: demographic information about the program and PD, 10 vignettes about hypothetical residents struggling in various ACGME milestones to assess how PDs would address these deficiencies, and self-reported PD preparedness and availability of resources to support struggling residents.

RESULTS

In total, we received 82 responses to our survey. All PDs who participated in our study reported having struggling residents in their program. The three most common ways struggling residents are identified were faculty word-of-mouth, formal evaluations such as milestones and ABSITE performance, and resident word-of-mouth. Over 18% of PDs reported having little to no relevant training in addressing the needs of a struggling resident, and 65.9% of PDs did not feel that their program had 'completely adequate' resources to address these needs. In the majority of cases, PDs offer mentorship with themselves or other faculty as a remediation strategy with infrequent use of other resources.

CONCLUSIONS

Strategies to identify struggling residents and remediation strategies varied widely across programs. Diversifying remediation approaches should be considered for more effective remediation.

Post graduate remediation programs in medicine: a scoping review

BACKGROUND

Recognizing that physicians may struggle to achieve knowledge, skills, attitudes and or conduct at one or more stages during their training has highlighted the importance of the 'deliberate practice of improving performance through practising beyond one's comfort level under guidance'. However, variations in physician, program, contextual and healthcare and educational systems complicate efforts to create a consistent approach to remediation. Balancing the inevitable disparities in approaches and settings with the need for continuity and effective oversight of the remediation process, as well as the context and population specific nature of remediation, this review will scrutinise the remediation of physicians in training to better guide the design, structuring and oversight of new remediation programs.

METHODS

Krishna's Systematic Evidence Based Approach is adopted to guide this Systematic Scoping Review (SSR in SEBA) to enhance the transparency and reproducibility of this review. A structured search for articles on remediation programs for licenced physicians who have completed their pre-registration postings and who are in training positions published between 1st January 1990 and 31st December 2021 in PubMed, Scopus, ERIC, Google Scholar, PsycINFO, ASSIA, HMIC, DARE and Web of Science databases was carried out. The included articles were concurrently thematically and content analysed using SEBA's Split Approach. Similarities in the identified themes and categories were combined in the Jigsaw Perspective and compared with the tabulated summaries of included articles in the Funnelling Process to create the domains that will guide discussions.

RESULTS

The research team retrieved 5512 abstracts, reviewed 304 full-text articles and included 101 articles. The domains identified were characteristics, indications, frameworks, domains, enablers and barriers and unique features of remediation in licenced physicians in training programs.

CONCLUSION

Building upon our findings and guided by Hauer et al. approach to remediation and Taylor and Hamdy's Multi-theories Model, we proffer a theoretically grounded 7-stage evidence-based remediation framework to enhance understanding of remediation in licenced physicians in training programs. We believe this framework can guide program design and reframe remediation's role as an integral part of training programs and a source of support and professional, academic, research, interprofessional and personal development.

How We Do It: Remediation Pathways in a Surgical Simulation Curriculum for Competency Improvement

OBJECTIVE

Validated assessment of procedural knowledge and skills with formative remediation is a foundational part of achieving surgical competency. High-fidelity simulation programs provide a unique area to assess resident proficiency and independence, as well as to assist in identifying residents in need of further practice. While several studies have validated the use of simulation to attain proficiency of specific technical skills, few have validated remediation pathways for their trainees objectively. In this descriptive analysis, we review 2 remediation pathways within our simulation training curricula and how these are used in assessments of resident proficiency.

MATERIALS AND METHODS

Two methods of remediation were formulated for use in high-fidelity simulation labs. One remediation pathway was a summative process, where ultimate judgment of resident competency was assessed through intra-operative assessments of a holistic skill set. The second remediation pathway was a formative "coaching" process, where feedback is given at several intervals along the pathway towards a specific technical skills competence. All general surgery residents are enrolled in the longitudinal, simulation curricula.

RESULTS

Approximately one-third of surgical residents entered into a remediation pathway for either of the high-fidelity simulation curricula. Both residents and faculty expressed support for the summative and formative remediation pathways as constructed. Residents who entered remediation pathways believed it was a beneficial exercise, and the most common feedback was that remediation principles should be expanded to all residents. Interestingly, faculty demonstrated stronger support for the formative coaching feedback model than the summative assessment model.

CONCLUSIONS

Through the complementary use of both formative and summative remediation pathways, resident competence can be enriched in a constructive, nonpunitive method for self-directed performance improvement. Both trainees and faculty express high satisfaction with programs explicitly organized to ensure that skills are rated through a standardized process.

Emotion in remediation: A scoping review of the medical education literature

OBJECTIVES

Remediation can be crucial and high stakes for medical learners, and experts agree it is often not optimally conducted. Research from other fields indicates that explicit incorporation of emotion improves education because of emotion's documented impacts on learning. Because this could present an important opportunity for improving remediation, we aimed to investigate how the literature on remediation interventions in medical education discusses emotion.

METHODS

The authors used Arksey and O'Malley's framework to conduct a scoping literature review of records describing remediation interventions in medical education, using PubMed, CINAHL Complete, ERIC, Web of Science and APA PsycInfo databases, including all English-language publications through 1 May 2020 meeting search criteria. They included publications discussing remediation interventions either empirically or theoretically, pertaining to physicians or physician trainees of any level. Two independent reviewers used a standardised data extraction form to report descriptive information; they reviewed included records for the presence of mentions of emotion, described the mentions and analysed results thematically.

RESULTS

Of 1644 records, 199 met inclusion criteria and were reviewed in full. Of those, 112 (56%) mentioned emotion in some way; others focused solely on cognitive aspects of remediation. The mentions of emotion fell into three themes based on when the emotion was cited as present: during regular coursework or practice, upon referral for remediation and during remediation. One-quarter of records (50) indicated potential intentional incorporation of emotion into remediation programme design, but they were non-specific as to how emotions related to the learning process itself.

CONCLUSION

Even though emotion is omnipresent in remediation, medical educators frequently do not factor emotion into the design of remediation approaches and rarely explicitly utilise emotion to improve the learning process. Applications from other fields may help medical educators leverage emotion to improve learning in remediation, including strategies to frame and design remediation.

Early Detection and Remediation of Problem Learners

There are myriad types of problem learners in surgical residency and most have difficulty in more than 1 competency. Programs that use a standard curriculum of study and assessment are most successful in identifying struggling learners early. Many problem learners lack appropriate systems for study; a multidisciplinary educational team that is separate from the team that evaluates the success of remediation is critical. Struggling residents who require formal remediation benefit from performance improvement plans that clearly outline the issues of concern, describe the steps required for remediation, define success of remediation, and outline consequences for failure to remediate appropriately.

Remediation of Underperformance in Surgical Trainees - A Scoping Review

OBJECTIVE

Surgical trainees with significant underperformance require formal support to return to an expected standard, termed remediation. The aim of this scoping review was to define remediation interventions, approaches, and contexts.

DESIGN

Following scoping review protocols, we set out to identify the evidence-base for remediation of surgical trainees, outline key concepts and uncover areas to stimulate further research.

RESULTS

From a screen of 80 articles, 24 reported on remediation of surgical trainees. Most were from medical journals (n = 21, 88%) and published in the United States (n = 20, 83%). Ten articles (41%) reported outcomes of remediation of a trainee cohort and 7 (19%) were survey reports from surgical directors. The remainder were a mix of commentaries, editorials or reviews. Thirteen articles (54%) described trainees with deficiencies in multiple competencies, 8 articles (33%) had a singular focus on academic performance, and 1 article (3%) on technical skills. All articles used targeted individualized remediation strategies, a range of intervention methods (some multimodal) and recommended a 6- to 12-month period of remediation (n = 7, 29%). The program director was often the only supervisor (n = 12, 50%). One article reported trainees' perspective of the process and one used educational theory to inform remediation.

CONCLUSIONS

Data with clearly reported outcomes were limited, but we found that targeted, individualized, multimodal and long-term remediation covering a range of competencies have been reported in the literature for surgical trainees. There is a need for development of explicit frameworks, strengthen the support for supervisors and trainees and further apply educational theory to develop better interventions that remediate deficiencies for all competencies.

Remediation programmes for practising doctors to restore patient safety: the RESTORE realist review

Background

An underperforming doctor puts patient safety at risk. Remediation is an intervention intended to address underperformance and return a doctor to safe practice. Used in health-care systems all over the world, it has clear implications for both patient safety and doctor retention in the workforce. However, there is limited evidence underpinning remediation programmes, particularly a lack of knowledge as to why and how a remedial intervention may work to change a doctor’s practice.

Objectives

To (1) conduct a realist review of the literature to ascertain why, how, in what contexts, for whom and to what extent remediation programmes for practising doctors work to restore patient safety; and (2) provide recommendations on tailoring, implementation and design strategies to improve remediation interventions for doctors.

Design

A realist review of the literature underpinned by the Realist And MEta-narrative Evidence Syntheses: Evolving Standards quality and reporting standards.

Data sources

Searches of bibliographic databases were conducted in June 2018 using the following databases: EMBASE, MEDLINE, Cumulative Index to Nursing and Allied Health Literature, PsycINFO, Education Resources Information Center, Database of Abstracts of Reviews of Effects, Applied Social Sciences Index and Abstracts, and Health Management Information Consortium. Grey literature searches were conducted in June 2019 using the following: Google Scholar (Google Inc., Mountain View, CA, USA), OpenGrey, NHS England, North Grey Literature Collection, National Institute for Health and Care Excellence Evidence, Electronic Theses Online Service, Health Systems Evidence and Turning Research into Practice. Further relevant studies were identified via backward citation searching, searching the libraries of the core research team and through a stakeholder group.

Review methods

Realist review is a theory-orientated and explanatory approach to the synthesis of evidence that seeks to develop programme theories about how an intervention produces its effects. We developed a programme theory of remediation by convening a stakeholder group and undertaking a systematic search of the literature. We included all studies in the English language on the remediation of practising doctors, all study designs, all health-care settings and all outcome measures. We extracted relevant sections of text relating to the programme theory. Extracted data were then synthesised using a realist logic of analysis to identify context–mechanism–outcome configurations.

Results

A total of 141 records were included. Of the 141 studies included in the review, 64% related to North America and 14% were from the UK. The majority of studies (72%) were published between 2008 and 2018. A total of 33% of articles were commentaries, 30% were research papers, 25% were case studies and 12% were other types of articles. Among the research papers, 64% were quantitative, 19% were literature reviews, 14% were qualitative and 3% were mixed methods. A total of 40% of the articles were about junior doctors/residents, 31% were about practicing physicians, 17% were about a mixture of both (with some including medical students) and 12% were not applicable. A total of 40% of studies focused on remediating all areas of clinical practice, including medical knowledge, clinical skills and professionalism. A total of 27% of studies focused on professionalism only, 19% focused on knowledge and/or clinical skills and 14% did not specify. A total of 32% of studies described a remediation intervention, 16% outlined strategies for designing remediation programmes, 11% outlined remediation models and 41% were not applicable. Twenty-nine context–mechanism–outcome configurations were identified. Remediation programmes work when they develop doctors’ insight and motivation, and reinforce behaviour change. Strategies such as providing safe spaces, using advocacy to develop trust in the remediation process and carefully framing feedback create contexts in which psychological safety and professional dissonance lead to the development of insight. Involving the remediating doctor in remediation planning can provide a perceived sense of control in the process and this, alongside correcting causal attribution, goal-setting, destigmatising remediation and clarity of consequences, helps motivate doctors to change. Sustained change may be facilitated by practising new behaviours and skills and through guided reflection.

Limitations

Limitations were the low quality of included literature and limited number of UK-based studies.

Future work

Future work should use the recommendations to optimise the delivery of existing remediation programmes for doctors in the NHS.

Study registration

This study is registered as PROSPERO CRD42018088779.

Funding

This project was funded by the National Institute for Health Research (NIHR) Health Services and Delivery Research programme and will be published in full in Health Services and Delivery Research; Vol. 9, No. 11. See the NIHR Journals Library website for further project information.

Remediation for surgical trainees: recommendations from a narrative review

BACKGROUND

Remediation involves formalized support for surgical trainees with significant underperformance to return to expected standards. There is a need to understand current evidence of remediation for surgical trainees to inform practice and justify investment of resources.

METHODS

Following the principles of a systematic review, we conducted a narrative analysis to make recommendations for remediation of underperforming surgical trainees.

RESULTS

From a review of 55 articles on remediation of trainees in medical and surgical sub-specialities, we have identified system and process level recommendations. Remediation is reported as long-term, complex and resource-intensive. Establishing a defined and standardized remediation framework enables co-ordination of multi-modal interventions. System level recommendations aim to consolidate protocols via developing better assessment, intervention and re-evaluation modalities whilst also strengthening support to supervisors conducting the remediation. Process level recommendations should be tailored for the specific needs of each trainee, aiming to be proactive with interventions within a programmatic framework. Regular reassessment is required, and long-term follow-up shows that remediation efforts are often successful.

CONCLUSION

While remediation within a programmatic framework is complex, it is often a successful approach to return surgical trainees to their expected standard. Future directions involve applying learning theories, encouraging research methods and to develop integrated collaborate protocols and support to synergize efforts.

The Surgical Knowledge "Growth Curve": Predicting ABSITE Scores and Identifying "At-Risk" Residents

OBJECTIVE

Resident performance on the American Board of Surgery In-Training Examination (ABSITE) is used for evaluation of surgical knowledge and guides resident selection for institutional remediation programs. Remediation thresholds have historically been based on ABSITE percentile scores; however, this does not account for predictors that can impact a resident's exam performance. We sought to identify predictors of yearly ABSITE performance to help identify residents "at-risk" for performing below their expected growth trajectory.

DESIGN

The knowledge of the residents, as measured by standardized ABSITE scores, was modeled as a function of the corresponding postgraduate year via a linear mixed effects regression model. Additional model covariates included written USMLE-1-3 examination scores, gender, number of practice questions completed, and percentage correct of practice questions. For each resident, the predicted ABSITE standard score along with a 95% bootstrap prediction interval was obtained. Both resident-specific and population-level predictions for ABSITE standard scores were also estimated.

SETTING

The study was conducted at a single, large academic medical center (Massachusetts General Hospital, Boston, MA).

PARTICIPANTS

Six years of general surgery resident score reports at a single institution between 2014 and 2019 were deidentified and analyzed.

RESULTS

A total of 376 score reports from 130 residents were analyzed. Covariates that had a significant effect on the model included USMLE-1 score (PGY1: p = 0.013; PGY2: p = 0.007; PGY3: p = 0.011), USMLE-2 score (PGY1: p < 0.001; PGY2: p < 0.001; PGY3: p < 0.001; PGY4: p < 0.001; PGY5: p = 0.032), male gender (PGY1: p = 0.003; PGY2: p < 0.001; PGY3: p < 0.001; PGY4: p = 0.008), and number of practice questions completed (p=0.003). Five residents were identified as having "fallen off" their predicted knowledge curve, including a single resident on 2 occasions. Population prediction curves were obtained at 7 different covariate percentile levels (5%, 10%, 25%, 50%, 75%, 90%, and 95%) that could be used to plot predicted resident knowledge progress.

CONCLUSION

Performance on USMLE-1 and -2 examinations, male gender, and number of practice questions completed were positive predictors of ABSITE performance. Creating residency-wide knowledge growth curves as well as individualized predictive ABSITE performance models allows for more efficient identification of residents potentially at risk for poor ABSITE performance and structured monitoring of surgical knowledge progression.

Principles of Remediation for the Struggling Neurosurgery Resident

BACKGROUND

When resident physicians fail to demonstrate appropriate milestone competencies early in their neurologic surgery residency, a plan of remediation is necessary.

METHODS

Once any psychologic, physical, or behavioral causes of identified knowledge or psychomotor deficiencies have been identified and addressed, the next step is to develop a plan to close these gaps. Specific areas that are assessed for deficits include medical knowledge, clinical reasoning and judgment, clinical skills, time management and organization, interpersonal skills, communication, and professionalism. Specific learning goals and objectives, as well as teaching and learning methods, pertain to the unique areas of deficit, and all of these must be considered with the goal of developing a resident-specific remediation plan.

RESULTS

A plan for assessment of the remediation process is described, including an evaluation of what constitutes individual resident remediation success.

CONCLUSIONS

Finally, a discussion of the prior resident remediation studies across many disciplines is made.

Remediation of learners struggling with communication skills: a systematic review

BACKGROUND

Communication skills is a core area of competency for healthcare practitioners. However, trainees deficient in those skills are not identified early enough to address the deficiency. Furthermore, faculty often struggle to identify effective remediation strategies for those who fail to meet expectations. We undertook a systematic review to determine which assessment methods are appropriate to identify learners that struggle with communication skills and the strategies used to remediate them.

METHODS

The literature was searched from January 1998 through to May 2019 using academic databases and grey literature. Trainees were defined as healthcare practitioners in undergraduate, graduate and continuing education. Characteristics of studies, assessment and intervention strategies and outcomes were synthesized qualitatively and summarized in tables.

RESULTS

From an initial 1636 records, 16 (1%) studies met the review criteria. Majority of the learners were medical students. A few studies (44%) included students from other disciplines, residents and physicians in practice. The remediation programs, in the studies, ranged from 1 week to 1 year. Around half of the studies focused solely on learners struggling with communication skills. The majority of studies used a format of a clinical OSCE to identify struggling learners. None of the studies had a single intervention strategy with the majority including an experiential component with feedback.

CONCLUSIONS

A few studies collectively described the diagnosis, remediation intervention and the assessment of the outcomes of remediation of communication skills. For a remediation strategy to be successful it is important to ensure: (i) early identification and diagnosis, (ii) the development of an individualized plan and (iii) providing reassessment with feedback to the learner.

Orthopaedic Resident Remediation: Frequency, Interventions, and Outcomes

Background

Despite multiple studies in other areas of surgical resident education, there is no current literature regarding orthopaedic resident remediation. The goal of the present study was to use a survey format to determine the frequency of remediation, the underlying etiology, the intervention strategies utilized, and the outcomes.

Methods

The program directors of all current 159 Accreditation Council for Graduate Medical Education-recognized orthopaedic residency programs in the United States were e-mailed a non-identifying digital survey. Follow-up e-mails were sent monthly for a 3-month period. The data remained anonymous, with no identifying information. The data are reported with use of descriptive statistics.

Results

Seventy (44%) of 159 program directors responded; most were from academic institutions with 4 to 5 residents per class. One hundred and fifty-eight residents were remediated, with the greatest number being remediated during the third postgraduate year (PGY-3). Professionalism, patient care, and communication were the most common deficiencies requiring remediation. Mentorship, feedback, and probation were the most common interventions. Of the 117 residents for which the outcome was reported, 58 graduated on time, 14 graduated from another program, 25 graduated from another specialty, 14 were terminated, 3 pursued litigation, and 3 chose a non-medical career. Rehabilitation, didactics, feedback, and mentorship were associated with the highest rates of on-time graduation.

Conclusions

Most remediated residents were PGY-3, suggesting increased scrutiny as residents moved from junior to senior responsibilities. The greatest number of deficiencies requiring remediation pertained to the affective domain, which highlights the importance of screening measures such as away rotations and interviews. The relationships formed during increased feedback sessions and mentorships can help problem residents to graduate on time.

Professionalism: A Core Competency, but What Does it Mean? A Survey of Surgery Residents

OBJECTIVE

Professionalism is 1 of the 6 core competencies of the Accreditation Council of Graduate Medical Education. Despite its obvious importance, it is poorly defined in the literature and an understanding of its meaning has not been evaluated on surgical trainees. The American College of Surgeons (ACS) has previously published tenets of surgical professionalism. However, surgery residents may not share similar views on professionalism as those of the ACS.

DESIGN

Surgical residents of all levels at 2 surgery residencies located in the same city were interviewed regarding their personal definitions, thoughts, and experiences regarding professionalism during their training. They were then queried regarding 20 points of professionalism as outlined by the ACS tenets of professionalism.

SETTING

The study utilized the surgery residencies at William Beaumont Army Medical Center and Texas Tech University Health Science Center in El Paso, Texas.

PARTICIPANTS

All general surgery residents at each program were invited to participate in the study. Eighteen residents volunteered to take the survey and be interviewed.

RESULTS

The definitions of professionalism centered on clinical competence. Surgery residents conveyed experiences with both professional and unprofessional behavior. Seven of the 20 ACS tenets of professionalism were unanimously agreed upon. There were key differences between resident definitions and those as outlined by the ACS. The least agreed upon ACS tenets of professionalism include professionalism education, public education, and public health.

CONCLUSIONS

Surgical trainees express personal experiences in both professional and unprofessional behavior. Their definitions of professionalism are not as expansive as those of the ACS and seem to focus on patient and colleague interaction. Due to the lack of congruency, a tailored curriculum for professionalism based upon ACS tenets appears warranted.

The Science of Selection: Using Best Practices From Industry to Improve Success in Surgery Training

BACKGROUND

The selection of high-quality applicants is critical to the future of surgery. However, it is unclear if current practices meet industry criteria of a successful selection system, as measured by administrative efficiency and performance and attrition of those selected.

METHODS

We performed a modified systematic review process to gain an understanding of current selection processes, remediation practices, and attrition rates in surgery residency training programs in the United States. We also conducted semistructured interviews with local residency program directors and coordinators to obtain a specific snapshot of the amount of time and resources dedicated to these activities in various sized programs. The associated financial costs of these activities are also presented.

RESULTS

The administrative costs for current residency selection processes are substantial, ranging from $45,000 to $148,000 for each program per year. Approximately 30% of residents require at least 1 remediation intervention, costing programs $3400 to $5300 per episode, and typically involve concerns around nontechnical skills. Attrition rates range from 20% to 40%.

CONCLUSIONS

This review suggests that additional methodologies may allow surgery residency programs to identify best-fit candidates more efficiently and effectively, while also decreasing remediation and attrition rates. Possible solutions include incorporation of structured interviews, personality inventories, and situational judgment tests. Resources dedicated to current interview practices, remediation efforts, and attrition management can be redirected to support these methodologies. By applying the science of selection and assessment to the recruitment process, programs may be able to make more data-driven decisions to identify candidates who will be successful at their institution.

Status of Resident Attrition From Surgical Residency in the Past, Present, and Future Outlook

OBJECTIVE

To investigate the current rate of attrition in general surgery residency, assess the risk factors, and identify prevention strategies.

DESIGN

A literature review of the PubMed and MEDLINE databases, from January 1, 1980 to February 1, 2016, for relevant articles. The calculated attrition rate and the statistically significant influencing factors were the main measures and outcomes.

SELECTION

All English language articles that described attrition from a general surgery residency were included. Articles that performed an assessment of attrition rates, academic performance, reasons for resident loss, and demographics were identified and data from these studies were collected. Random-effect meta-analysis and meta-regression based on a generalized mixed-effects model was performed.

RESULTS

A total of 26 studies were included. Reported attrition rates ranged from 2% to 30% over the course of residency training. Random-effect meta-analysis is indicative of a yearly attrition rate of 2.4% (95% CI: 1.3%-3.5%) and a cumulative 5-year attrition rate of 12.9% (95% CI: 7.9%-17.8%). Most of them leave residency during their first 2 years, and the rate significantly decreases with increasing postgraduate year (p < 0.0001). The Accreditation Council for Graduate Medical Education mandated 80-hour week is associated with a higher rate, though not significantly (3.2% [95% CI: 1.3%-5.1%] vs. 2.2% [0.9%-3.5%], p = 0.37). Pooled analysis demonstrates no statistically significant difference in the rate of attrition between males and females (2.1% [95% CI: 1.1%-3%] vs. 2.9% [95% CI: 1.6%-4.1%], p = 0.73). Most remain in graduate medical education and pursue residency training in other specialties.

CONCLUSION

Attrition in general surgery most commonly occurs within the first 2 years of training and, in contrast to previous findings, is not related to female sex. Restrictions on work hours seem to have increased the rate, whereas remediation practices can prevent it. Training programs should direct efforts towards attrition-prevention strategies.

Effect of Performance Deficiencies on Graduation and Board Certification Rates: A 10-yr Multicenter Study of Anesthesiology Residents

BACKGROUND

This multicenter, retrospective study was conducted to determine how resident performance deficiencies affect graduation and board certification.

METHODS

Primary documents pertaining to resident performance were examined over a 10-yr period at four academic anesthesiology residencies. Residents entering training between 2000 and 2009 were included, with follow-up through February 2016. Residents receiving actions by the programs' Clinical Competency Committee were categorized by the area of deficiency and compared to peers without deficiencies.

RESULTS

A total of 865 residents were studied (range: 127 to 275 per program). Of these, 215 residents received a total of 405 actions from their respective Clinical Competency Committee. Among those who received an action compared to those who did not, the proportion graduating differed (93 vs. 99%, respectively, P < 0.001), as did the proportion achieving board certification (89 vs. 99%, respectively, P < 0.001). When a single deficiency in an Essential Attribute (e.g., ethical, honest, respectful behavior; absence of impairment) was identified, the proportion graduating dropped to 55%. When more than three Accreditation Council for Graduate Medical Education Core Competencies were deficient, the proportion graduating also dropped significantly.

CONCLUSIONS

Overall graduation and board certification rates were consistently high in residents with no, or isolated, deficiencies. Residents deficient in an Essential Attribute, or multiple competencies, are at high risk of not graduating or achieving board certification. More research is needed on the effectiveness and selective deployment of remediation efforts, particularly for high-risk groups.

The Fundamentals of Resident Dismissal

Residents have the rights and responsibilities of both students and employees. Dismissal of a resident from a training program is traumatic and has lasting repercussions for the program director, the faculty, the dismissed resident, and the residency. A review of English language literature was performed using PUBMED and OVID databases, using the search terms, resident dismissal, resident termination, student dismissal, student and resident evaluation, legal aspects of education, and remediation. The references of each publication were also reviewed to identify additional appropriate citations. If the Just Cause threshold has been met, educators have the absolute discretion to evaluate academic and clinical performance. Legal opinion has stated that it is not necessary to wait until a patient is harmed to dismiss a resident. Evaluations should be standard and robust. Negative evaluations are not defamatory as the resident gave consent to be evaluated. Provided departmental and institutional polices have been followed, a resident can be dismissed without a formal hearing. Residencies are entitled to modify academic requirements and dismissal is not considered a breach of contract. Although there is anxiety regarding resident dismissal, the courts have uniformly supported faculty having this role. When indicated, failure to dismiss a resident also places the program director and the faculty at risk for educational malpractice.

The problem resident behavior guide: strategies for remediation

In 2012, the ACGME supplemented the core competencies with outcomes-based milestones for resident performance within the six competency domains. These milestones address the knowledge, skills, abilities, attitudes, and experiences that a resident is expected to progress through during the course of training. Even prior to the initiation of the milestones, there was a paucity of EM literature addressing the remediation of problem resident behaviors and there remain few readily accessible tools to aid in the implementation of a remediation plan. The goal of the "Problem Resident Behavior Guide" is to provide specific strategies for resident remediation based on deficiencies identified within the framework of the EM milestones. The "Problem Resident Behavior Guide" is a written instructional manual that provides concrete examples of remediation strategies to address specific milestone deficiencies. The more than 200 strategies stem from the experiences of the authors who have professional experience at three different academic hospitals and emergency medicine residency programs, supplemented by recommendations from educational leaders as well as utilization of valuable education adjuncts, such as focused simulation exercises, lecture preparation, and themed ED shifts. Most recommendations require active participation by the resident with guidance by faculty to achieve the remediation expectations. The ACGME outcomes-based milestones aid in the identification of deficiencies with regards to resident performance without providing recommendations on remediation. The Problem Resident Behavior Guide can therefore have a significant impact by filling in this gap.

The threat of the dyscompetent resident: A plea to make the implicit more explicit!

Although several examples of frameworks dealing with students' unprofessional behaviour are available, guidance on how to deal locally or regionally with dysfunctional residents is limited (Hickson et al. in Acad Med 82(11):1040-1048, 2007b; Leape and Fromson in Ann Intern Med 144(2):107-115, 2006). Any 'rules' are mostly unwritten, and often emerge by trial and error within the specialty training programme (Stern and Papadakis in N Engl J Med 355(17):1794-1799, 2006). It is nevertheless of utmost importance that objectives, rules and guidelines comparable to those existing in undergraduate training (Project Team Consilium Abeundi van Luijk in Professional behaviour: teaching, assessing and coaching students. Final report and appendices. Mosae Libris, 2005; van Mook et al. in Neth J Crit Care 16(4):162-173, 2010a) are developed for postgraduate training. And that implicit rules are made explicit. This article outlines a framework based on the lessons learned from contemporary postgraduate medical training programmes.

What does remediation and probation status mean? A survey of emergency medicine residency program directors

OBJECTIVES

Emergency medicine (EM) residency program directors (PDs) nationwide place residents on remediation and probation. However, the Accreditation Council for Graduate Medical Education and the EM PDs have not defined these terms, and individual institutions must set guidelines defining a change in resident status from good standing to remediation or probation. The primary objective of this study was to determine if EM PDs follow a common process to guide actions when residents are placed on remediation and probation.

METHODS

An anonymous electronic survey was distributed to EM PDs via e-mail using SurveyMonkey to determine the current practice followed after residents are placed on remediation or probation. The survey queried four designations: informal remediation, formal remediation, informal probation, and formal probation. These designations were compared for deficits in the domains of medical knowledge (MK) and non-MK remediation. The survey asked what process for designation exists and what actions are triggered, specifically if documentation is placed in a resident's file, if the graduate medical education (GME) office is notified, if faculty are informed, or if resident privileges are limited. Descriptive data are reported.

RESULTS

Eighty-one of 160 PDs responded. An official policy on remediation and/or probation was reported by 41 (50.6%) programs. The status of informal remediation is used by 73 (90.1%), 80 (98.8%) have formal remediation, 40 (49.4%) have informal probation, and 79 (97.5%) have formal probation. There was great variation among PDs in the management and definition of remediation and probation. Between 81 and 86% of programs place an official letter into the resident's file regarding formal remediation and probation. However, only about 50% notify the GME office when a resident is placed on formal remediation. There were no statistical differences between MK and non-MK remediation practices.

CONCLUSIONS

There is significant variation among EM programs regarding the process of remediation and probation. The definition of these terms and the actions triggered are variable across programs. Based on these findings, suggestions toward a standardized approach for remediation and probation in GME programs are provided.

Surgery residency curriculum examination scores predict future American Board of Surgery in-training examination performance

IMPORTANCE

A protected block curriculum (PBC) with postcurriculum examinations for all surgical residents has been provided to assure coverage of core curricular topics. Biannual assessment of resident competency will soon be required by the Next Accreditation System.

OBJECTIVE

To identify opportunities for early medical knowledge assessment and interventions, we examined whether performance in postcurriculum multiple-choice examinations (PCEs) is predictive of performance in the American Board of Surgery In-Training Examination (ABSITE) and clinical service competency assessments.

DESIGN

Retrospective single-institutional education research study.

SETTING

Academic general surgery residency program.

PARTICIPANTS

A total of 49 surgical residents.

INTERVENTION

Data for PGY1 and PGY2 residents participating in the 2008 to 2012 PBC are included. Each resident completed 6 PCEs during each year.

MAIN OUTCOME MEASURES

The results of 6 examinations were correlated to percentage-correct ABSITE scores and clinical assessments based on the 6 Accreditation Council for Graduate Medical Education core competencies. Individual ABSITE performance was compared between PGY1 and PGY2. Statistical analysis included multivariate linear regression and bivariate Pearson correlations.

RESULTS

A total of 49 residents completed the PGY1 PBC and 36 completed the PGY2 curriculum. Linear regression analysis of percentage-correct ABSITE and PCE scores demonstrated a statistically significant correlation between the PGY1 PCE 1 score and the subsequent PGY1 ABSITE score (p = 0.037, β = 0.299). Similarly, the PGY2 PCE 1 score predicted performance in the PGY2 ABSITE (p = 0.015, β = 0.383). The ABSITE scores correlated between PGY1 and PGY2 with statistical significance, r = 0.675, p = 0.001. Performance on the 6 Accreditation Council for Graduate Medical Education core competencies correlated between PGY1 and PGY2, r = 0.729, p = 0.001, but did not correlate with PCE scores during either years.

CONCLUSIONS AND RELEVANCE

Within a mature PBC, early performance in a PGY1 and PGY2 PCE is predictive of performance in the respective ABSITE. This information can be used for formative assessment and early remediation of residents who are predicted to be at risk for poor performance in the ABSITE.

Assessment of surgical training

BACKGROUND

The patient safety imperative has raised expectations regarding the responsibility of medical educators and decision makers to ensure that physicians are competent. Ensuring that trainees are ready for independent practice upon graduation is challenged by reduced work hours such that trainees spend less time in the OR and perform fewer cases than desirable.

METHODS

The literature on the assessment of technical and non--technical operative skills and professionalism was reviewed in order to make recommendations to identify barriers to evaluation.

DISCUSSION

Barriers to documenting performance deficiencies include uncertainty as to what should be documented, and concerns about the negative impact of critical evaluations on faculty popularity. Additional challenges include a lack of clear standards for performance and effective remediation options.

CONCLUSIONS

Trainee performance should be evaluated in a rigorous, reliable and meaningful way to ensure that graduates have the skills necessary for safe, independent practice.

Remediation practices in Canadian psychiatry clerkships

OBJECTIVES

There is a lack of information regarding interventions for undergraduate students at Canadian medical schools who require remediation during their psychiatric training. The need for a theoretical framework to guide remediation has been identified. In this study, we sought to characterize remediation taking place in undergraduate psychiatry education, particularly during clerkship. A secondary goal was to ascertain whether those responsible for remediation were aware of the stages of change (transtheoretical) model, and whether they formally incorporated this framework (or another) into their remediation processes.

METHODS

A short six-question survey on remediation practices was e-mailed to educators responsible for undergraduate psychiatry education at all 17 Canadian medical schools as identified through the Canadian Organization of Undergraduate Psychiatry Educators (COUPE).

RESULTS

The response rate was 67 %. Respondents' overall impressions of their school's remediation process were that it was "highly effective" (25 %), or "somewhat effective" (67 %); 8 % reported being unsure of its effectiveness. While 75 % of survey respondents were aware of the stages of change model, only 17 % reported using this framework: no alternate theoretical frameworks were reported.

CONCLUSIONS

The most common form of evaluation was multiple-choice question (MCQ) exams, and the most common form of remediation was exam rewrites. There is little information regarding the long-term outcomes of remediation, and further research would be useful in formulating recommendations regarding best practices.

Remediation in the context of the competencies: a survey of pediatrics residency program directors

BACKGROUND

The 6 competencies defined by the Accreditation Council for Graduate Medical Education provide the framework of assessment for trainees in the US graduate medical education system, but few studies have investigated their impact on remediation.

METHODS

We obtained data via an anonymous online survey of pediatrics residency program directors. For the purposes of the survey, remediation was defined as "any form of additional training, supervision, or assistance above that required for a typical resident." Respondents were asked to quantify 3 groups of residents: (1) residents requiring remediation; (2) residents whose training was extended for remediation purposes; and (3) residents whose training was terminated owing to issues related to remediation. For each group, the proportion of residents with deficiencies in each of the 6 competencies was calculated.

RESULTS

In all 3 groups, deficiencies in medical knowledge and patient care were most common; deficiencies in professionalism and communication were moderately common; and deficiencies in systems-based practice and practice-based learning and improvement were least common. Residents whose training was terminated were more likely to have deficiencies in multiple competencies.

CONCLUSION

Although medical knowledge and patient care are reported most frequently, deficiencies in any of the 6 competencies can lead to the need for remediation in pediatrics residents. Residents who are terminated are more likely to have deficits in multiple competencies. It will be critical to develop and refine tools to measure achievement in all 6 competencies as the graduate medical education community may be moving further toward individualized training schedules and competency-based, rather than time-based, training.

Remediation of residents in difficulty: a retrospective 10-year review of the experience of a postgraduate board of examiners

PURPOSE

To determine, through a 10-year review, (1) the prevalence of residents in difficulty, (2) characteristics of these residents, (3) areas of residents' weakness, and (4) outcomes of residents who undergo remediation.

METHOD

A retrospective review of resident records for the University of Toronto Faculty of Medicine's (UT-FOM) Board of Examiners for Postgraduate Programs (BOE-PG) was done from July 1, 1999 to June 30, 2009 using predetermined data elements entered into a standardized form and analyzed for trends and significance. Outcomes for residents in difficulty were tracked through university registration systems and licensure databases.

RESULTS

During 10 years, 103 UT-FOM residents were referred to the BOE-PG, representing 3% of all residents enrolled. The annual prevalence of residents referred to the BOE-PG ranged from 0.2% to 1.5%. The CanMEDS framework was used to classify areas of residents' weaknesses and organize remediation plans. All 100 residents studied had either medical expertise (85%) or professionalism (15%) weaknesses or both. Residents had difficulties with an average of 2.6 CanMEDS Roles, with highest frequencies of Medical Expert (85%) Professional (51%), Communicator (49%), Manager (43%), and Collaborator (20%). Often, there were multiple remediation periods, with an average of six months' duration. Usually, remediation was successful; 78% completed residency education, 17% were unsuccessful, and 5% remained in training.

CONCLUSION

Residents in difficulty have multiple areas of weakness. The CanMEDS framework is an effective approach to classifying problems and designing remediation plans. Successful completion of residency education after remediation is the most common outcome.

The utility of the ABS in-training examination (ABSITE) score forms: percent correct and percentile score in the assessment of surgical residents

OBJECTIVE

The American Board of Surgery (ABS) provides program directors with ABS In-Training Examination (ABSITE) scores in the following forms: Percent correct score and percentile score. It is of interest to note how residency programs utilize the different forms of ABSITE scores in assessment of surgical residents for progression in training. We conducted a survey of program directors to ascertain the present situation.

METHODS

A structured questionnaire was sent to all program director members of the Association of Program Directors in Surgery.

RESULTS

114/210 program directors (54%) answered the survey. To assess residents, 3 programs used only the percentage correct score, 23 programs used only the percentile score, and 88 programs used both scores. The majority (70/89 or 79%) of the programs used a 30th percentile score as the minimum passing score. 88/111 (79%) programs had a remedial process for residents with poor performance on ABSITE. 60 percent of the programs had never used poor ABSITE performance to defer individual resident promotion. Programs that used ABSITE performance for remediation and deferral of promotion did it based on percentile score rather than percent correct score. Program directors felt that the better indicator of a resident's knowledge and progression in surgical residency was percent correct score (42%) vs percentile score (32%), while 10% felt that neither was an adequate indicator.

CONCLUSIONS

ABSITE score is being used as one of the measures to assess residents. Programs need to ensure that an effective remedial process is in place to assist residents with poor performance.

Guiding principles for resident remediation: recommendations of the CORD remediation task force

Remediation of residents is a common problem and requires organized, goal-directed efforts to solve. The Council of Emergency Medicine Residency Directors (CORD) has created a task force to identify best practices for remediation and to develop guidelines for resident remediation. Faculty members of CORD volunteered to participate in periodic meetings, organized discussions and literature reviews to develop overall guidelines for resident remediation and in a collaborative authorship of this article identifying best practices for remediation. The task force recommends that residency programs: 1. Make efforts to understand the challenges of remediation, and recognize that the goal is successful correction of deficits, but that some deficits are not remediable. 2. Make efforts aimed at early identification of residents requiring remediation. 3. Create objective, achievable goals for remediation and maintain strict adherence to the terms of those plans, including planning for resolution when setting goals for remediation. 4. Involve the institution's Graduate Medical Education Committee (GMEC) early in remediation to assist with planning, obtaining resources, and documentation. 5. Involve appropriate faculty and educate those faculty into the role and terms of the specific remediation plan. 6. Ensure appropriate documentation of all stages of remediation. Resident remediation is frequently necessary and specific steps may be taken to justify, document, facilitate, and objectify the remediation process. Best practices for each step are identified and reported by the task force.

Resident evaluation and remediation: a comprehensive approach

BACKGROUND

A comprehensive evaluation and remediation program is an essential component of any residency program. The evaluation system should identify problems accurately and early and allow residents with problems to be assigned to a remediation program that effectively deals with them. Elements of a proactive remediation program include a process for outlining deficiencies, providing resources for improvement, communicating clear goals for acceptable performance, and reevaluating performance against these goals.

INTERVENTION

In recognition of the importance of early detection and prompt remediation of the struggling resident, we sought to develop a multifaceted approach to resident evaluation with the aim of early identification and prompt remediation of difficulties. This article describes our comprehensive evaluation program and remediation program, which uses resources within our radiology department and institutional graduate medical education office.

DISCUSSION

An effective evaluation system should identify problems accurately and early, whereas a proactive remediation program should effectively deal with issues once they are identified.