A Qualitative Evaluation of a Clinical Faculty Mentorship Program Using a Realist Evaluation Approach

Perspectives on Professional Development Among University and Community Pediatric Hospitalists

Multiple professional societies have emphasized the importance of professional development for physicians. This qualitative study aimed to explore pediatric hospitalists' perceptions of professional development needs and to refine a framework for professional development in pediatric hospital medicine (PHM). We conducted four focus groups in April to May 2019 with 19 pediatric hospitalists at six clinical sites within a single institution. Participants identified key components of professional development including skill development, personal growth, career satisfaction, and individualization. Hospitalists agreed upon 8 domains of professional development: clinical excellence, advocacy, global health, health care administration, informatics, medical education, quality improvement, and research. They also identified missing the mentorship necessary to change their passions into career advancement, highlighted barriers and facilitators, and noted that an alignment in personally meaningful projects to what is meaningful to the institution was in everyone's best interests. Faculty programs should build infrastructure to aid pediatric hospitalists in achieving their career goals.

Evaluation of Continuing Professional Development for Physicians - Time for Change: A Scoping Review

Introduction

Evaluation of education interventions is essential for continuous improvement as it provides insights into how and why outcomes occur. Specifically, for physicians' continuing professional development (CPD) programs, which aim to upskill physicians in a range of practice-essential domains, evaluations are crucial to assure physicians' continuous development, enhanced patient care and safety. However, evaluations of health professions education (HPE) interventions tend to be outcomes focused, failing to capture how and why outcomes occur. This scoping review aimed to identify evaluation techniques used to evaluate CPD programs for physicians, and to determine how these techniques are being implemented as well as the their quality.

Methods

We searched PubMed, Embase, Web of Science, among others for English publications on evaluation of CPD programs for physicians, in the past decade. We used a data charting template to extract study details regarding the evaluation techniques and produced a checklist to assess the quality of the evaluations.

Results

101 studies were included; of which 91 studies did not use an evaluation framework. Our findings revealed shortcomings in the evaluations of CPD programs including lack of attention to: intervention processes; unintended outcomes and contextual factors; use of theory; evaluation framework use; and rationale for chosen evaluation method.

Discussion

Our findings highlighted major gaps in the evaluation techniques employed in physicians' CPD. Attention needs to be paid to evaluating both program processes and outcomes to illuminate how and why impacts are or are not occurring.

Realist approach to community-based participatory research on a community health break-down in Japan: mechanism reasoning, knowledge and a trust partnership

BACKGROUND

Our 10-year programme of community health regeneration and community-based participatory research (CBPR) was initially unknown. However, we succeeded in creating a collaboration between residents, medical staff, and administrative staff. We adopted a realist approach as an evaluation method.

METHODS

The realist approach evaluates a programme using a Context-Mechanism-Outcome configuration (CMOc), which is a relatively new methodology. First, the programme manager summarised the entire programme, conducted questionnaires and interviews with seven core members, and summarised each into a CMOc. The programme was evaluated with particular focus on mechanistic reasoning.

RESULTS

The number of doctors and nurses increased and residents became more active. The success factors were the acquisition of participants' knowledge and trust partnerships. In addition, it was important that the timing of the activity was good and that the participants were highly conscious.

CONCLUSIONS

The 10-year CBPR was examined using a realist evaluation method. Knowledge acquisition and trust partnerships are important for reasoning mechanism.

What Really Matters for Supervision Training Workshops? A Realist Evaluation

PURPOSE

Supervision training supports health care supervisors to perform their essential functions. Realist evaluations are increasingly popular for evaluating complex educational interventions, but no such evaluations exist appraising supervision workshops. Building on an earlier realist synthesis of supervision training, the authors evaluated whether supervision workshops work, for whom and under what circumstances, and why.

METHOD

The authors conducted a 2-stage realist evaluation during 2018-2019 to refine and develop program theory. The intervention involved half-day, face-to-face supervision workshops as part of an Australian state-wide government-funded program for health care and human services supervisors. Data collection involved realist interviews with 10 workshop developers (stage 1) and 43 supervisors (stage 2). The authors employed team-based data analysis using realist logic to refine and develop program theory by identifying contexts, mechanisms, outcomes, and context-mechanism-outcome configurations.

RESULTS

Despite their brevity, the supervision workshops had many reported benefits for supervisors (e.g., improved satisfaction) through various perceived mechanisms pertaining to pedagogy (e.g., mixed pedagogies), workshops (e.g., optimal duration), and individuals (e.g., supervisor engagement). However, they also yielded negative reported outcomes (e.g., suboptimal knowledge gains) brought about by assorted perceived mechanisms related to pedagogy (e.g., suboptimal peer learning), workshops (e.g., content irrelevance), and individuals (e.g., suboptimal facilitator competence). Such mechanisms were thought to be triggered by diverse contexts including supervisors' levels of experience, sector, and workplace supervision cultures.

CONCLUSIONS

While the findings partly support the realist synthesis of supervision training and previous realist evaluations of faculty development, this realist evaluation extends this literature considerably. Health care educators should employ mixed pedagogies (e.g., didactic teaching, peer learning), relevant content, optimal workshop duration, and competent/engaging facilitators. Educators also need to tailor workshops according to supervisors' contexts including the sectors and supervision cultures in which supervision is practiced, and supervisors' levels of experience (e.g., experienced supervisors appreciated workshop brevity).

The role of mentoring, supervision, coaching, teaching and instruction on professional identity formation: a systematic scoping review

BACKGROUND

Mentoring's pivotal role in nurturing professional identity formation (PIF) owes much to its combined use with supervision, coaching, tutoring, instruction, and teaching. However the effects of this combination called the 'mentoring umbrella' remains poorly understood. This systematic scoping review thus aims to map current understanding.

METHODS

A Systematic Evidence-Based Approach guided systematic scoping review seeks to map current understanding of the 'mentoring umbrella' and its effects on PIF on medical students and physicians in training. It is hoped that insights provided will guide structuring, support and oversight of the 'mentoring umbrella' in nurturing PIF. Articles published between 2000 and 2021 in PubMed, Scopus, ERIC and the Cochrane databases were scrutinised. The included articles were concurrently summarised and tabulated and concurrently analysed using content and thematic analysis and tabulated. The themes and categories identified were compared with the summaries of the included articles to create accountable and reproducible domains that guide the discussion.

RESULTS

A total of 12201 abstracts were reviewed, 657 full text articles evaluated, and 207 articles included. The three domains identified were definitions; impact on PIF; and enablers and barriers. The mentoring umbrella shapes PIF in 3 stages and builds a cognitive base of essential knowledge, skills and professional attitudes. The cognitive base informs thinking, conduct and opinions in early supervised clinical exposure in Communities of practice (COP). The COPs' individualised approach to the inculcation of desired professional characteristics, goals, values, principles and beliefs reshapes the individual's identity whilst the socialisation process sees to their integration into current identities.

CONCLUSION

The mentoring umbrella's provides personalised longitudinal support in the COP and socialisation process. Understanding it is key to addressing difficulties faced and ensuring holistic and timely support.

Unpacking economic programme theory for supervision training: Preliminary steps towards realist economic evaluation

INTRODUCTION

Cost studies are increasingly popular given resource constraints. While scholars stress the context-dependent nature of cost, and the importance of theory, cost studies remain context-blind and atheoretical. However, realist economic evaluation (REE) privileges context and the testing/refinement of economic programme theory. This preliminary REE serves to test and refine economic programme theory for supervision training programmes of different durations to better inform future programme design/implementation.

METHODS

Our preliminary REE unpacked how short (half-day) and extended (12 week) supervision training programmes in Victoria, Australia, produced costs and outcomes. We employed mixed methods: qualitative realist and quantitative cost methods. Economically optimised programme models were developed guided by identified cost-sensitive mechanisms and contexts.

RESULTS

As part of identified context-mechanism-outcome configurations (CMOCs) for both training programmes, we found a wider diversity of positive outcomes but greater costs for the extended programme (11 outcomes; AU$3069/learner) compared with the short programme (7 outcomes; $385/learner). We identified four shared cost-sensitive mechanisms for both programmes (training duration, learner protected time, learner engagement, and facilitator competence) and one shared cost-sensitive context (learners' supervisory experience). We developed two economically optimised programme models: (1) short programme for experienced supervisors facilitated by senior educators ($406/learner); and (2) extended programme for inexperienced supervisors facilitated by junior educators ($2104/learner).

DISCUSSION

Our initial economic programme theory was somewhat supported, refuted and refined. Results were partly consistent with previous research, but also extended it through unpacking cost-sensitive mechanisms and contexts. Although our preliminary REE fills a pressing gap in the methodology literature, conducting REE was challenging given our desire to integrate economic and realist analyses fully, and remain faithful to realist principles. Attention to training duration and experience levels of the facilitator-learner dyad may help to balance the cost and outcomes of training programmes.

The effectiveness of extended-duration supervision training for nurses and allied health professionals: A realist evaluation

BACKGROUND

Training helps maintain high-quality supervision and its associated benefits (e.g. reduced burnout, improved care). While studies have previously evaluated extended-duration supervision training programmes, none have treated these as complex interventions so have not employed realist approaches.

OBJECTIVES

Building on a previous realist synthesis, this evaluation tests and develops programme theory for extended-duration supervision training to answer the question: to what extent does the supervision training programme work, for whom, under what circumstances and why?

DESIGN

We conducted a realist evaluation of a novel state-wide Victorian 3-month supervision training programme including one or two 3.5-h workshops followed by weekly reflexive longitudinal audio diaries (LADs) for up to 12 weeks.

METHODS

Realist evaluation data comprised 25 entrance interviews with nurses and allied health professionals, 176 LADs (and 29 written diaries), and 23 exit interviews. We employed team-based realist analysis to identify context-mechanism-outcome configurations (CMOCs) to test and develop programme theory.

RESULTS

We refined four recurring CMOCs from the realist synthesis programme theory, found insufficient evidence for two, and established five new recurring CMOCs. We identified multiple positive outcomes from our extended-duration supervision training programme (e.g. improved supervisor practices) through various mechanisms relating to pedagogy (e.g. weekly reflexivity), supervisors (e.g. engagement), and workplaces (e.g. enabling supervision cultures). Some negative outcomes were reported (e.g. decreased engagement) through various mechanisms (e.g. suboptimal training design). Such mechanisms were thought to come about by diverse contexts including supervisors (e.g. inexperienced/experienced), professions (nursing/allied health), and organisations (supervision-enabled/non-enabled cultures).

CONCLUSIONS

Our findings extend the realist synthesis programme theory by highlighting various contexts triggering outcome-generating mechanisms. Programme outcomes are maximised through ongoing supervisor reflexivity paying attention to facilitator-supervisor relationships, as well as protected time for supervisors to translate learning into practice.

An evaluation of the current mentorship/preceptorship practices for newly qualified radiographers in Northern Ireland

INTRODUCTION

Mentorship/Preceptorship (M/P) has been utilised within the nursing profession since the early 1980's. Successful, structured M/P programmes can be hugely beneficial to Northern Ireland (NI) Trusts who recruit regularly and often rely on the fluidity of staff movement regionally. In the absence of standardised tools to accurately and universally measure the competency of newly qualified Radiographers (NQR) as they evolve, establishing the benchmark for effective practice within Radiology departments in NI is difficult and highly subjective at best. This study aimed to evaluate the current M/P strategies within NI as perceived by NQR and Radiology Managers (RM).

METHODS

A mix of both qualitative and quantitative data was obtained using questionnaires through a scoping exercise. Opinions were sought from a target audience of NQR, who began full-time employment following graduation in 2018, and RM involved in the delivery of current M/P programmes within the NI Trusts.

RESULTS

Responses were gained from all five NI trusts to achieve a representative regional sample, with final opinions of RM (n = 8, 54%) and NQR (n = 30, 67%) received. With the exception of one response, all NQRs confirmed receiving some form of M/P support within their inaugural post.

CONCLUSION

Significant variations were apparent in both the structure and delivery of Trust M/P induction programmes, calling into question the comparable competency of NQRs regionally.

IMPLICATIONS FOR PRACTICE

The disparity in approach towards M/P programmes across NI, and subsequent lack of comparability of NQR competence, endorses the development of a more robust and universal method for the regional assessment of NQRs, such as that of the 'Flying Start NHS®' programme utilised by National Health Service (NHS) Scotland, in combination with knowledge and skills framework (KSF) practices in supervision.