Would I Trust or Will I Trust? The Gap between Entrustment Determinations and Entrustment Decisions for Trainees in Pharmacy and Other Health Professions

Assessment of Entrustable Professional Activities Among Dutch Endocrine Supervisors

Entrustable Professional Activities (EPAs) are an important tool to support individualisation of medical training in a competency-based setting and are increasingly implemented in the clinical speciality training for endocrinologist. This study aims to assess interrater agreement and factors that potentially impact EPA scores. Five known factors that affect entrustment decisions in health profesions training (capability, integrity, reliability, humility, agency) were used in this study. A case-vignette study using standardised written cases. Case vignettes (n = 6) on the topics thyroid disease, pituitary disease, adrenal disease, calcium and bone disorders, diabetes mellitus, and gonadal disorders were written by two endocrinologists and a medical education expert and assessed by endocrinologists experienced in the supervision of residents in training. Primary outcome is the inter-rater agreement of entrustment decisions for endocrine EPAs among raters. Secondary outcomes included the dichotomous interrater agreement (entrusted vs. non-entrusted), and an exploration of factors that impact decision-making. The study protocol was registered and approved by the Ethical Review Board of the Netherlands Association for Medical Education (NVMO-ERB # 2020.2.5). Nine endocrinologists from six different academic regions participated. Overall, the Fleiss Kappa measure of agreement for the EPA level was 0.11 (95% CI: 0.03-0.22) and for the entrustment decision 0.24 (95% CI 0.11-0.37). Of the five features that impacted the entrustment decision, capability was ranked as the most important by a majority of raters (56%-67%) in every case. There is a considerable discrepancy between the EPA levels assigned by different raters. These findings emphasise the need to base entrustment decisions on multiple observations, made by a team of supervisors and enriched with factors other than direct medical competence.

Do summative entrustment decisions actually lead to entrustment?

BACKGROUND

Entrustable professional activities (EPAs) were introduced across Dutch postgraduate programmes between 2017 and 2019. We aimed to understand the extent to which residents actually were granted increased clinical responsibility upon receiving summative entrustment for an EPA, a critical feature of its use.

METHODS

A survey study was conducted among all Dutch residents who started dermatology training in 2018 and 2019 and all Dutch dermatology programme directors (PDs). We chose an EPA designed for early entrustment in residency (identification, treatment and care regarding a simple dermatological problem in the ambulatory setting). The survey contained two hypothetical clinical cases that aligned with this EPA. The questions were aimed to determine whether and when residents should request supervision. Similar questions were posed to PDs.

FINDINGS

Twenty four residents (56%) and 19 PDs (79%) completed the survey. The majority of the residents (65%) and PDs (63%) confirmed that competent dermatology residents (level 4) are generally allowed to perform EPA1 unsupervised, particularly when seeing patients from GPs. However, still a substantial proportion of the level 4 residents, working in University Medical Centers (36%) indicated that they had to request supervision in the assessment of these patients. For 2nd opinions, the results were typically the opposite.

DISCUSSION AND CONCLUSION

This study demonstrated that, at least in one specialty and one country, the introduction of EPAs and entrustment decision making procedure generally led to the intended autonomy of the resident.

Medical competence as a multilayered construct

BACKGROUND

The conceptualisation of medical competence is central to its use in competency-based medical education. Calls for 'fixed standards' with 'flexible pathways', recommended in recent reports, require competence to be well defined. Making competence explicit and measurable has, however, been difficult, in part due to a tension between the need for standardisation and the acknowledgment that medical professionals must also be valued as unique individuals. To address these conflicting demands, a multilayered conceptualisation of competence is proposed, with implications for the definition of standards and approaches to assessment.

THE MODEL

Three layers are elaborated. This first is a core layer of canonical knowledge and skill, 'that, which every professional should possess', independent of the context of practice. The second layer is context-dependent knowledge, skill, and attitude, visible through practice in health care. The third layer of personalised competence includes personal skills, interests, habits and convictions, integrated with one's personality. This layer, discussed with reference to Vygotsky's concept of Perezhivanie, cognitive load theory, self-determination theory and Maslow's 'self-actualisation', may be regarded as the art of medicine. We propose that fully matured professional competence requires all three layers, but that the assessment of each layer is different.

IMPLICATIONS

The assessment of canonical knowledge and skills (Layer 1) can be approached with classical psychometric conditions, that is, similar tests, circumstances and criteria for all. Context-dependent medical competence (Layer 2) must be assessed differently, because conditions of assessment across candidates cannot be standardised. Here, multiple sources of information must be merged and intersubjective expert agreement should ground decisions about progression and level of clinical autonomy of trainees. Competence as the art of medicine (Layer 3) cannot be standardised and should not be assessed with the purpose of permission to practice. The pursuit of personal excellence in this level, however, can be recognised and rewarded.