The development of a framework of entrustable professional activities for the intern year in Ireland

Development and Validation of Entrustable Professional Activities for Provisionally Registered (Intern) Pharmacists in Australia

OBJECTIVE

The purpose of this study was to describe the development and validation of entrustable professional activities (EPAs) for a pharmacy intern training program in Australia.

METHODS

Performance outcomes with EPAs suggested as an assessment in the intern year were identified and EPAs were developed. The EQual rubric, designed for EPAs in graduate medical education settings, was used to validate the EPAs. Eight subject matter experts were recruited to validate 14 EPAs using prespecified cutoffs in the EQual rubric to determine the degree of alignment with established key domains of the EPA construct. Descriptive statistics for the EQual rubric scores were calculated for each of the 14 EPAs overall and for each of the 3 domains. Any EPA falling below the prespecified cut score required revisions and a second round of validation.

RESULTS

Eight responses were recorded for the first round of validation. Seven EPAs were below the discrete units of work cutoff (4.17); 2 of these EPAs were also below the EPA curricular role cutoff (4.00). All scores described in the EPAs as entrustable, essential, and important tasks of the profession were above the cutoff (4.00). Seven subject matter experts responded to the second round of validation. All 7 revised EPAs scored above the EQual rubric's overall cutoff of 4.07.

CONCLUSION

Fourteen EPAs were validated for use with provisionally registered (intern) pharmacists. To our knowledge, this is the first time that the EQual rubric has been used in the evaluation and validation of pharmacy EPAs.

Development and validation of core entrustable professional activities for abdominal radiology

OBJECTIVES

To develop and validate European entrustable professional activities (EPAs) for sub-specialised hepatobiliary and gastrointestinal (HB/GI) diagnostic imaging.

MATERIALS AND METHODS

Both European Society of Radiology and national curricula in HB/GI diagnostic radiology were thoroughly reviewed, resulting in preliminary EPAs drafted by a pilot group of expert radiologists in 2 different countries. Each EPA was fully described with 7 components (Specification/limitations; Potential risks of failing; Relevant domains of competence; Required experience, knowledge, skills, attitude and behaviour; Assessment information sources to assess progress and ground a summative entrustment decision; Entrustment for which level of supervision is to be reached; and Expiration date). The modified Delphi method with 3 Delphi rounds was chosen for validation. Content validity index (CVI) and median values were used for validation.

RESULTS

There were 15 preliminary EPAs, some of them divided according to 2 levels: resident and fellow level. The 37 members of the Delphi group were based in 2 different European countries with a background experience of 10 represented countries. Subsequent to the first Delphi round, 6 EPAs were accepted (CVI ≥ 0.8, median ≥ 4), 6 needed major revisions (CVI 0.7-0.79, median ≥ 4), 3 were rejected (CVI < 0.7) and 1 was added. After the second Delphi round, both the 6 revised EPAs and the additional one met the validation criteria (CVI ≥ 0.8, median ≥ 4). Finally, 13 EPAs were validated during the 3rd Delphi round with an agreement percentage of 95-100%.

CONCLUSION

This study creates and validates EPAs for sub-specialised HB/GI diagnostic imaging.

CRITICAL RELEVANCE STATEMENT

Thirteen EPAs for sub-specialised hepatobiliary and gastrointestinal diagnostic imaging were created with a strong methodology, and as a first example set in sub-specialised diagnostic imaging, they provide a template for others to be created.

KEY POINTS

• The competence-based teaching in medical studies has recently been reintroduced through EPAs. • Thirteen EPAs have been developed for hepatobiliary and gastrointestinal sub-specialised diagnostic imaging. • These EPAs were validated using a Delphi modified method and provide a template for other to be created.

Effectiveness of a proficiency-based progression e-learning approach to training in communication in the context of clinically deteriorating patients: a multi-arm randomised controlled trial

OBJECTIVE

To determine the effectiveness of proficiency-based progression (PBP) e-learning in training in communication concerning clinically deteriorating patients.

DESIGN

Single-centre multi-arm randomised double-blind controlled trial with three parallel arms.

RANDOMISATION, SETTING AND PARTICIPANTS

A computer-generated program randomised and allocated 120 final year medical students in an Irish University into three trial groups.

INTERVENTION

Each group completed the standard Identification, Situation, Background, Assessment, Recommendation communication e-learning; group 1 Heath Service Executive course group (HSE) performed this alone; group 2 (PBP) performed additional e-learning using PBP scenarios with expert-determined proficiency benchmarks composed of weighted marking schemes of steps, errors and critical errors cut-offs; group 3 (S) (self-directed, no PBP) performed additional e-learning with identical scenarios to (PBP) without PBP.

MAIN OUTCOME MEASURES

Primary analysis was based on 114 students, comparing ability to reach expert-determined predefined proficiency benchmark in standardised low-fidelity simulation assessment, before and after completion of each group's e-learning requirements. Performance was recorded and scored by two independent blinded assessors.

RESULTS

Post-intervention, proficiency in each group in the low-fidelity simulation environment improved with statistically significant difference in proficiency between groups (p<0.001). Proficiency was highest in (PBP) (81.1%, 30/37). Post hoc pairwise comparisons revealed statistically significant differences between (PBP) and self-directed (S) (p<0.001) and (HSE) (p<0.001). No statistically significant difference existed between (S) and (HSE) (p=0.479). Changes in proficiency from pre-intervention to post-intervention were significantly different between the three groups (p=0.001). Post-intervention, an extra 67.6% (25/37) in (PBP) achieved proficiency in the low-fidelity simulation. Post hoc pairwise comparisons revealed statistically significant differences between (PBP) and both (S) (p=0.020) and (HSE) (p<0.001). No statistically significant difference was found between (S) and (HSE) (p=0.156).

CONCLUSIONS

PBP e-learning is a more effective way to train in communication concerning clinically deteriorating patients than standard e-learning or e-learning without PBP.

TRIAL REGISTRATION NUMBER

NCT02937597.

A Review of Entrustable Professional Activities in Pharmacy Education

Objective. To review the published literature describing how entrustable professional activities (EPAs) are currently used in health professions education with a focus on pharmacy education.Findings. English-language searches in the databases CINAHL Plus and MEDLINE were conducted for January 2011 through March 2021, which identified 21 publications for inclusion in this review. The following themes were identified: frameworks for the development of EPAs, the implementation and evaluation of EPAs for workplace learning, and gaps in knowledge and future directions for EPAs.Summary. Currently, no standardized approach exists for developing EPA frameworks for health disciplines. Also, the ways that EPAs are implemented and evaluated as an educational tool in practice settings are inconsistent. An opportunity exists to further establish the development, implementation, and evaluation of EPAs in all clinical practice settings. Much can be learned from other health professions to ensure successful translation of EPAs into pharmacy education.

Developing entrustable professional activities for family medicine training in South Africa

Workplace-based assessment (WPBA) is becoming part of high-stake assessments in specialist training. Entrustable professional activities (EPAs) are a recent addition to WPBA. This is the first South African publication on developing EPAs for postgraduate family medicine training. An EPA is a unit of practice, observable in the workplace, constituting several tasks with underlying knowledge, skills and professional behaviours. Entrustable professional activities allow for entrustable decisions regarding competence in a described work context. A national workgroup representing all nine postgraduate training programmes in South Africa has developed 19 EPAs. This new concept needs change management to understand the theory and practice of EPAs. Family medicine departments with large clinical workloads are small, necessitating navigating logistical issues to develop EPAs. It has unmasked existing workplace learning and assessment challenges.Contribution: This article contributes new thinking to developing EPAs for family medicine in an effort to understand more authentic WPBA nationally.

Developing a CPD model for Eswatini-a participatory action research study

BACKGROUND

Continuing professional development (CPD) is a key aspect to fulfil a commitment to lifelong learning for professionals registered with the Medical and Dental Council, the intention being to promote the health of patients and develop clinical expertise. The absence of formal CPD requirements for practitioners in Eswatini has resulted in a move to introduce an accredited system.

METHODS

The qualitative study followed a participatory action research (PAR) methodology using a cooperative inquiry group of 10 medical practitioners in Eswatini to investigate how the current CPD program could be improved and formalised. PAR entailed four stages; observation, reflection, planning and action, using a semi-structured format to explore the areas of concern.

RESULTS

Reflecting on the current situation resulted in three ways to improve CPD being identified: (1) adopt a formal, compulsory CPD model; (2) recognise achievements by practitioners who endeavour to improve their skills/knowledge through Entrustable Professional Activities, and (3) ensure that CPD is relevant to the workplace by using Quality-Improvement CPD (QI-CPD) and reflective diaries. These would be done by involving local practitioners, using adult learning principles and ensuring continuous evaluation and improvement of the CPD model.

CONCLUSIONS

There was general agreement on the need for a formalised CPD system to improve skill levels and provide an open platform to enhance patient care in a resource constrained setting. The findings provided information that can be used to plan and action its implementation through engagement with the country's doctors in various forums and through ongoing research.

Evaluating the Impact of Supervision on Surgical Trainees Stress Response During Simulated Surgical Procedures; A Crossover Randomized Trial

OBJECTIVE

The aim of this study was to evaluate the cumulative impact of supervision on technical skills and surrogate stress markers in surgical trainees.

DESIGN

This was a quasi-experimental crossover study to evaluate the impact of attending supervision on orthopedic trainee stress response during a simulated surgical procedure. Enrolled residents performed a proximal femoral nail module with the Precision OS system twice; once independently, and once under direct attending supervision, whilst wearing a heart rate monitor. Mean and maximum heart rates were recorded. Simulated performance was assessed using validated simulator-based metrics. Student's t-test was used to evaluate the impact of supervision on trainee heart rate, and performance ranking.

SETTING

Tertiary trauma center in a Regional Orthopedic Unit PARTICIPANTS: Orthopedic interns and residents within our institution were invited to participate, with 20 participants included for analysis.

RESULTS

Both supervised and unsupervised mean heart rate was significantly higher (p = 0.001) than baseline recorded heart rates. Supervised mean and maximum HR were significantly higher than unsupervised HR during module completion (p = 0.015; p = 0.001). Calories burned demonstrated correlation to surrogate stress markers, significantly higher in supervised sessions (p = 0.004). Performance metrics demonstrated superior performance in senior-level participants, with a decrement in performance during supervision, failing to reach significance.

CONCLUSION

The development of accretion of technical and non-technical skills required in surgical training pathways may derive benefit from the use of simulation-based training in surgical residents with both supervised and unsupervised sessions.

The Role of Entrustable Professional Activities in Competency-based Medical Education for Anesthesiology Residents: A Pilot Phase

Background

After graduation, physicians should be able to provide professional and safe services without the need for supervision by their clinical professors, mandating a competency-based medical education (CBME) approach.

Objectives

This study aimed to develop a national model of entrustable professional activities (EPAs) based on our experiences in the Department of Anesthesiology and Critical Care (DACC), Shahid Beheshti University of Medical Sciences (SBMU).

Methods

The primary EPA design plan was designed in a 10-step model as a career roadmap for the project. The texts were prepared according to a consensus-based approach. On the other hand, the texts were reviewed and revised by a broad team of faculty in a daily workshop.

Results

The final product included 14 topics for EPA as the first round of targeting topics for anesthesiology residents. The texts were developed using previous studies and were standardized considering national standards.

Conclusions

We described a clear path toward designing and implementing EPAs in anesthesiology residency programs to improve the quality of the graduated residents. Though the basic theory is the same, each country needs its formula for implementing the process.

Longitudinal survey of self-reported level of entrustment across the first year of clinical practice

Background. Newly graduated medical students often report that they lack the skills required to care for patients, and feel unprepared for clinical practice. However, little is known about when, and if, they acquire these skills in practice. The aim of this study was to assess self-reported level of entrustment in, and frequency of performance of, the seven Entrustable Professional Activities (EPAs) comprising the EPA framework for interns in Ireland. EPAs describe discrete activities that are essential to a particular profession. Methods. Self-report survey data were collected from doctors in the Republic of Ireland during their first year of clinical practice at four different time points during 2020/21. Results. Response rates to the survey varied from 73.3% (126/172) at Time 1 to 25.6% (44/172) at Time 4. After three months, the respondents reported that they could execute all of the EPAs, inclusive of 12 essential procedural skills, with indirect supervision. As the year progressed there was an increase in the proportion of respondents reporting that they performed the EPAs at least once a week. However, the proportion of respondents performing five of the essential procedural skills (e.g. nasogastric tube insertion) remained low across all time points. Conclusion. Consideration should be given as to how to better prepare medical students to execute these EPAs, how the interns can be better supported during the first quarter of internship. The findings from this research are positive. However, there is an urgent need to carry out formal assessments of entrustability, rather than relying on self-report.