Early predictors of need for remediation in the Australian general practice training program: a retrospective cohort study

Unprofessional behaviour of GP residents leading to a dismissal dispute: characteristics and outcomes of those who appeal

BACKGROUND

Recognition of poor performance in General Practice trainees is important because underperformance compromises patients' health and safety. However, in General Practice, research on persistent underperformance while in training and its ultimate consequences is almost completely lacking. We aim to explore the unprofessional behaviours of residents in General Practice who were dismissed from training and who litigated against dismissal.

METHODS

We performed a structured analysis using open-source data from all General Practice cases before the Conciliation Board of the Royal Dutch Medical Association between 2011 and 2020. Anonymised law cases about residents from all Dutch GP training programmes were analysed in terms of the quantitative and qualitative aspects related to performance.

RESULTS

Between 2011 and 2020, 24 residents who were dismissed from training challenged their programme director's decision. Dismissed residents performed poorly in several competencies, including communication, medical expertise and most prominently, professionalism. Over 90% of dismissed residents failed on professionalism. Most lacked self-awareness and/or failed to profit from feedback. Approximately 80% failed on communication, and about 60% on medical expertise as well. A large majority (more than 80%) of dismissed residents had previously participated in some form of remediation.

CONCLUSIONS

Deficiencies in both professionalism and communication were the most prevalent findings among the dismissed General Practice residents. These two deficiencies overlapped considerably. Dismissed residents who challenged their programme director's decision were considered to lack self-awareness, which requires introspection and the appreciation of feedback from others.

Dutch dismissal practices: characteristics, consequences, and contrasts in residents' case law in community-based practice versus hospital-based specialties

BACKGROUND

In the Netherlands, 2 to 10% of the residents terminate training prematurely. Infrequently, termination of training is by dismissal. Incidentally, residents may disagree, dispute and challenge these decisions from the programme directors. Resident dismissal is always a difficult decision, most commonly made after, repeated assessments, and triangulation of the resulting assessment data and one or more remediation attempts. Nevertheless, the underlying reasons for dismissal and the policies for remediation and dismissal may differ between training programmes. Such differences may however impact the chance of remediation success, the chance of dismissal and subsequent residents' appeals.

METHOD

We included a total of 70 residents from two groups (community-based and hospital-based specialties) during 10 years of appeals. Subsequently, we compared these groups on factors potentially associated with the outcome of the conciliation board decision regarding the residents' dismissal. We focused herein on remediation strategies applied, and reasons reported to dismiss residents.

RESULTS

In both groups, the most alleged reason to dismiss residents was lack of trainability, > 97%. This was related to deficiencies in professionalism in community-based practice and medical expertise in hospital-based specialties respectively. A reason less frequently mentioned was endangerment of patient care, < 26%. However, none of these residents accused of endangerment, actually jeopardized the patients' health, probably due to the vigilance of their supervisors. Remediation strategies varied between the two groups, whereas hospital-based specialties preferred formal remediation plans in contrast to community-based practice. A multitude of remediation strategies per competency (medical expertise, professionalism, communication, management) were applied and described in these law cases.

DISCUSSION

Residents' appeals in community-based practice were significantly less likely to succeed compared to hospital-based specialties. Hypothesised explanatory factors underlying these differences include community-based practices' more prominent attention to the longitudinal assessment of professionalism, the presence of regular quarterly progress meetings, precise documentation of deficiencies, and discretion over the timing of dismissal in contrast to dismissal in the hospital-based specialties which is only formally possible during scheduled formal summative assessment meetings.

Colleague appraisal of Australian general practitioners in training: an analysis of multisource feedback data

BACKGROUND

Multisource feedback is an evidence-based and validated tool used to provide clinicians, including those in training, feedback on their professional and interpersonal skills. Multisource feedback is mandatory for participants in the Royal Australian College of General Practitioners Practice Experience Program and for some Australian General Practice Training Registrars. Given the recency of the Practice Experience Program, there are currently no benchmarks available for comparison within the program and to other comparable cohorts including doctors in the Australian General Practice Training program. The aim of this study is to evaluate and compare colleague feedback within and across General Practice trainee cohorts.

METHODS

Colleague feedback, from multisource feedback of Practice Experience Program participants and Australian General Practice Training Registrars, collected between January 2018 and April 2020, was compared to identify similarities and differences. Analyses entailed descriptive statistics, between and within groups rater consistency and agreement measures, principal component analysis, t-tests, analysis of variance, and psychometric network analysis.

RESULTS

Colleague ratings of Practice Experience Program participants (overall average 88.58%) were lower than for Registrars (89.08%), although this difference was not significant. 'Communication with patients' was rated significantly lower for Practice Experience Program participants (2.13%) while this group was rated significantly better for their 'Ability to say no' (1.78%). Psychometric network analyses showed stronger linkages between items making up the behavioural component (compared to the items of the performance and self-management components, as found by principal component analysis) for Practice Experience Program participants as compared to Registrars. Practice Experience Program participants were stronger in clinical knowledge and skills as well as confidentiality, while Registrars were stronger in communicating with patients, managing their own stress, and in their management and leadership skills.

CONCLUSIONS

The multisource feedback scores of doctors undertaking the Practice Experience Program suggests that, while all mean values are 'very good' to 'excellent', there are areas for improvement. The linkages between skills suggests that Practice Experience Program doctors' skills are somewhat isolated and have yet to fully synthesise. We now have a better understanding of how different groups of General Practitioners in training compare with respect to professional and interpersonal skills. Based on the demonstrated differences, the Practice Experience Program might benefit from the addition of educational activities to target the less developed skills.

Post graduate remediation programs in medicine: a scoping review

BACKGROUND

Recognizing that physicians may struggle to achieve knowledge, skills, attitudes and or conduct at one or more stages during their training has highlighted the importance of the 'deliberate practice of improving performance through practising beyond one's comfort level under guidance'. However, variations in physician, program, contextual and healthcare and educational systems complicate efforts to create a consistent approach to remediation. Balancing the inevitable disparities in approaches and settings with the need for continuity and effective oversight of the remediation process, as well as the context and population specific nature of remediation, this review will scrutinise the remediation of physicians in training to better guide the design, structuring and oversight of new remediation programs.

METHODS

Krishna's Systematic Evidence Based Approach is adopted to guide this Systematic Scoping Review (SSR in SEBA) to enhance the transparency and reproducibility of this review. A structured search for articles on remediation programs for licenced physicians who have completed their pre-registration postings and who are in training positions published between 1st January 1990 and 31st December 2021 in PubMed, Scopus, ERIC, Google Scholar, PsycINFO, ASSIA, HMIC, DARE and Web of Science databases was carried out. The included articles were concurrently thematically and content analysed using SEBA's Split Approach. Similarities in the identified themes and categories were combined in the Jigsaw Perspective and compared with the tabulated summaries of included articles in the Funnelling Process to create the domains that will guide discussions.

RESULTS

The research team retrieved 5512 abstracts, reviewed 304 full-text articles and included 101 articles. The domains identified were characteristics, indications, frameworks, domains, enablers and barriers and unique features of remediation in licenced physicians in training programs.

CONCLUSION

Building upon our findings and guided by Hauer et al. approach to remediation and Taylor and Hamdy's Multi-theories Model, we proffer a theoretically grounded 7-stage evidence-based remediation framework to enhance understanding of remediation in licenced physicians in training programs. We believe this framework can guide program design and reframe remediation's role as an integral part of training programs and a source of support and professional, academic, research, interprofessional and personal development.

Identifying the at-risk General Practice trainee: a retrospective cohort meta-analysis of General Practice registrar flagging

A central principle of programmatic assessment is that the final decision is not a surprise to the learner. To achieve this, assessments must demonstrate predictive and consequential validity, however, to date, research has only focussed on the former. The present study attempts to address this gap by examining the predictive and consequential validity of flagging systems used by Australian General Practice regional training organisations (RTOs) in relation to Fellowship examinations. Informed by unstructured interviews with Senior Medical Educators to understand the flagging system of each RTO, meta-analyses of routinely-collected flagging data were used to examine the predictive validity of flagging at various points in training and exam performance. Additionally, flagging system features identified from the interviews were used to inform exploratory subgroup analyses and meta-regressions to further assess the predictive and consequential validity of these systems. Registrars flagged near the end of their training were two to four times more likely to fail Fellowship exams than their non-flagged counterparts. Regarding flagging system features, having graded (i.e. ordinal) flagging systems was associated with higher accuracy, whilst involving the assigned medical educator in remediation and initiating a formal diagnostic procedure following a flag improved registrars' chances of passing exams. These results demonstrate both predictive and consequential validity of flagging systems. We argue that flagging is most effective when initiated early in training in conjunction with mechanisms to maximise diagnostic accuracy and the quality of remediation programs.

Study protocol: content and perceived educational utility of different modalities of clinical teaching visit (CTV) workplace-based assessments within Australian general practice vocational training: a cross-sectional study

INTRODUCTION

Clinical teaching visits (CTVs) are formative workplace-based assessments that involve a senior general practitioner (GP) observing a clinical practice session of a general practice registrar (specialist vocational GP trainee). These visits constitute a key part of Australian GP training. Despite being mandatory and resource-intensive, there is a paucity of evidence regarding the content and educational utility of CTVs. This study aims to establish the content and educational utility of CTVs across varying practice settings within Australia, as perceived by registrars and their assessors ('CT visitors'). In addition, this study aims to establish registrar, CT visitor and practice factors associated with CTV content and perceived CTV utility ratings.

METHODS AND ANALYSIS

This study will collect data prospectively using online questionnaires completed soon after incident CTVs. Participants will be registrars and CT visitors of CTVs conducted from March 2020 to January 2021. The setting is three Regional Training Organisations across four Australian states and territories (encompassing 37% of Australian GP registrars).Outcome factors will be a number of specified CTV content elements occurring during the CTV as well as participants' perceptions of CTV utility, which will be analysed using univariate and multivariable regression.

ETHICS AND DISSEMINATION

Ethics approval has been granted by the University of Newcastle Human Research Ethics Committee, approval number H-2020-0037. Study findings are planned to be disseminated via conference presentation, peer-reviewed journals, educational practice translational workshops and the GP Synergy research subwebsite.

Remediation for surgical trainees: recommendations from a narrative review

BACKGROUND

Remediation involves formalized support for surgical trainees with significant underperformance to return to expected standards. There is a need to understand current evidence of remediation for surgical trainees to inform practice and justify investment of resources.

METHODS

Following the principles of a systematic review, we conducted a narrative analysis to make recommendations for remediation of underperforming surgical trainees.

RESULTS

From a review of 55 articles on remediation of trainees in medical and surgical sub-specialities, we have identified system and process level recommendations. Remediation is reported as long-term, complex and resource-intensive. Establishing a defined and standardized remediation framework enables co-ordination of multi-modal interventions. System level recommendations aim to consolidate protocols via developing better assessment, intervention and re-evaluation modalities whilst also strengthening support to supervisors conducting the remediation. Process level recommendations should be tailored for the specific needs of each trainee, aiming to be proactive with interventions within a programmatic framework. Regular reassessment is required, and long-term follow-up shows that remediation efforts are often successful.

CONCLUSION

While remediation within a programmatic framework is complex, it is often a successful approach to return surgical trainees to their expected standard. Future directions involve applying learning theories, encouraging research methods and to develop integrated collaborate protocols and support to synergize efforts.

Early predictors of summative assessment performance in general practice post-graduate training: A retrospective cohort study

BACKGROUND

Success in summative general practice (GP) training assessments is one indicator of competence for practice. Early-training factors predictive of outcomes would facilitate targeted interventions aimed at preventing candidate failures.

METHODS

We undertook a retrospective cohort study of Australian GP trainees in two training organizations over five years. Associations of pre-training and early-training predictors with summative examination scores in an Applied Knowledge Test (AKT), Key Features Paper (KFP), and Objective Structured Clinical Examination (OSCE), plus failure on any one of these, were tested via univariate and multivariable regression. Predictors were program-entry selection-score decile, pre-training-commencement multiple choice assessment (MCQA), direct observation of practice performance, and clinical supervisor reports.

RESULTS

On univariate analyses, selection decile and MCQA were associated with all outcomes except AKT. There were no associations of other predictors with any outcomes. On multivariable analysis, selection decile and MCQA performance were predictive of OSCE performance. MCQA performance was also predictive of KFP performance. On multivariable analysis, no predictors were associated with AKT performance or failing any examination.

CONCLUSIONS

Selection decile and pre-commencement MCQA performance were predictive of performance in some summative assessments. These findings could inform selection policies and targeted early interventions for trainees at most risk for exam failure.