1
|
Hatala R, Ellaway RH. Does authentic assessment undermine authentic learning? ADVANCES IN HEALTH SCIENCES EDUCATION : THEORY AND PRACTICE 2024:10.1007/s10459-024-10361-z. [PMID: 39102165 DOI: 10.1007/s10459-024-10361-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 08/06/2024]
Abstract
In this editorial the editors consider the ideals and realities of high and low stakes assessments in clinical workplaces, the impact of these assessments on clinical workplace learning, and the clash between authenticity in assessment and authenticity in learning.
Collapse
|
2
|
Gauthier S, Hatala R. The best of both worlds: Assessing trainee progression in the era of competency based medical education. MEDICAL EDUCATION 2024; 58:769-771. [PMID: 38597239 DOI: 10.1111/medu.15390] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/01/2024] [Accepted: 03/04/2024] [Indexed: 04/11/2024]
Abstract
The authors suggest reducing the use of WBA where it is not fit for purpose and developing locally sustainable and defensible programs of assessment as steps towards unlocking the value of CBME.
Collapse
|
3
|
van Enk A, MacDonald G, Hatala R, Gingerich A, Tam J. Not in the file: How competency committees work with undocumented contributions. MEDICAL EDUCATION 2024. [PMID: 38899368 DOI: 10.1111/medu.15457] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/22/2023] [Revised: 04/25/2024] [Accepted: 05/24/2024] [Indexed: 06/21/2024]
Abstract
INTRODUCTION Competence committees (CCs) centre their work around documentation of trainees' performance; undocumented contributions (i.e. informal, unrecorded material like personal judgements, experiential anecdotes and contextual information) evoke suspicion even though they may play a role in decision making. This qualitative multiple case study incorporates insights from a social practice perspective on writing to examine the use of undocumented contributions by the CCs of two large post-graduate training programmes, one in a more procedural (MP) speciality and the other in a less procedural (LP) one. METHODS Data were collected via observations of meetings and semi-structured interviews with CC members. In the analysis, conversations were organised into triptychs of lead-up, undocumented contribution(s), and follow-up. We then created thick descriptions around the undocumented contributions, drawing on conversational context and interview data to assign possible motivations and significance. RESULTS We found no instances in which undocumented contributions superseded the contents of a trainee's file or stood in for missing documentation. The number of undocumented contributions varied between the MP CC (six instances over two meetings) and the LP CC (22 instances over three meetings). MP CC discussions emphasised Entrustable Professional Activity (EPA) observations, whereas LP CC members paid more attention to narrative data. The divergent orientations of the CCs-adding an 'advis[ing]/guid[ing]' role versus focusing simply on evaluation-offers the most compelling explanation. In lead-ups, undocumented contributions were prompted by missing and flawed documentation, conflicting evidence and documentation at odds with members' perceptions. Recognising other 'red flags' in documentation often required professional experience. In follow-ups, purposes served by undocumented contributions varied with context and were difficult to generalise; we, therefore, provide deeper analysis of two vignettes to illustrate. CONCLUSIONS Our data suggest undocumented contributions often serve best efforts to ground decisions in documentation. We would encourage CC practices and policies be rooted in more nuanced approaches to documentation.
Collapse
|
4
|
Ginsburg S, Stroud L, Brydges R, Melvin L, Hatala R. Dual purposes by design: exploring alignment between residents' and academic advisors' documents in a longitudinal program. ADVANCES IN HEALTH SCIENCES EDUCATION : THEORY AND PRACTICE 2024:10.1007/s10459-024-10318-2. [PMID: 38438699 DOI: 10.1007/s10459-024-10318-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/27/2023] [Accepted: 02/04/2024] [Indexed: 03/06/2024]
Abstract
Longitudinal academic advising (AA) and coaching programs are increasingly implemented in competency based medical education (CBME) to help residents reflect and act on the voluminous assessment data they receive. Documents created by residents for purposes of reflection are often used for a second, summative purpose-to help competence committees make decisions-which may be problematic. Using inductive, thematic analysis we analyzed written comments generated by 21 resident-AA dyads in one large internal medicine program who met over a 2 year period to determine what residents write when asked to reflect, how this aligns with what the AAs report, and what changes occur over time (total 109 resident self-reflections and 105 AA reports). Residents commented more on their developing autonomy, progress and improvement than AAs, who commented far more on performance measures. Over time, residents' writing shifted away from intrinsic roles, patient care and improvement towards what AAs focused on, including getting EPAs (entrustable professional activities), studying and exams. For EPAs, the emphasis was on getting sufficient numbers rather than reflecting on what residents were learning. Our findings challenge the practice of dual-purposing documents, by questioning the blurring of formative and summative intent, the structure of forms and their multiple conflicting purposes, and assumptions about the advising relationship over time. Our study suggests a need to re-evaluate how reflective documents are used in CBME programs. Further research should explore whether and how documentation can best be used to support resident growth and development.
Collapse
|
5
|
Farrell L, Cuncic C, Hartford W, Hatala R, Ajjawi R. Goal co-construction and dialogue in an internal medicine longitudinal coaching programme. MEDICAL EDUCATION 2023; 57:265-271. [PMID: 36181337 DOI: 10.1111/medu.14942] [Citation(s) in RCA: 9] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/30/2022] [Revised: 08/18/2022] [Accepted: 09/28/2022] [Indexed: 06/16/2023]
Abstract
BACKGROUND Longitudinal coaching in residency programmes is becoming commonplace and requires iterative and collaborative discussions between coach and resident, with the shared development of goals. However, little is known about how goal development unfolds within coaching conversations over time and the effects these conversations have. We therefore built on current coaching theory by analysing goal development dialogues within resident and faculty coaching relationships. METHODS This was a qualitative study using interpretive description methodology. Eight internal medicine coach-resident dyads consented to audiotaping coaching meetings over a 1-year period. Transcripts from meetings and individual exit interviews were analysed thematically using goal co-construction as a sensitising concept. RESULTS Two themes were developed: (i) The content of goals discussed in coaching meetings focused on how to be a resident, with little discussion around challenges in direct patient care, and (ii) co-construction mainly occurred in how to meet goals, rather than in prioritising goals or co-constructing new goals. CONCLUSIONS In analysing goal development in the coach-resident relationships, conversations focused mainly around how to manage as a resident rather than how to improve direct patient care. This may be because academic coaching provides space separate from clinical work to focus on the stage-specific professional identity development of a resident. Going forward, focus should be on how to optimise longitudinal coaching conversations to ensure co-regulation and reflection on both clinical competencies and professional identity formation.
Collapse
|
6
|
Pusic MV, Cook DA, Friedman JL, Lorin JD, Rosenzweig BP, Tong CK, Smith S, Lineberry M, Hatala R. Modeling Diagnostic Expertise in Cases of Irreducible Uncertainty: The Decision-Aligned Response Model. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2023; 98:88-97. [PMID: 36576770 PMCID: PMC9780042 DOI: 10.1097/acm.0000000000004918] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 06/17/2023]
Abstract
PURPOSE Assessing expertise using psychometric models usually yields a measure of ability that is difficult to generalize to the complexity of diagnoses in clinical practice. However, using an item response modeling framework, it is possible to create a decision-aligned response model that captures a clinician's decision-making behavior on a continuous scale that fully represents competing diagnostic possibilities. In this proof-of-concept study, the authors demonstrate the necessary statistical conceptualization of this model using a specific electrocardiogram (ECG) example. METHOD The authors collected a range of ECGs with elevated ST segments due to either ST-elevation myocardial infarction (STEMI) or pericarditis. Based on pilot data, 20 ECGs were chosen to represent a continuum from "definitely STEMI" to "definitely pericarditis," including intermediate cases in which the diagnosis was intentionally unclear. Emergency medicine and cardiology physicians rated these ECGs on a 5-point scale ("definitely STEMI" to "definitely pericarditis"). The authors analyzed these ratings using a graded response model showing the degree to which each participant could separate the ECGs along the diagnostic continuum. The authors compared these metrics with the discharge diagnoses noted on chart review. RESULTS Thirty-seven participants rated the ECGs. As desired, the ECGs represented a range of phenotypes, including cases where participants were uncertain in their diagnosis. The response model showed that participants varied both in their propensity to diagnose one condition over another and in where they placed the thresholds between the 5 diagnostic categories. The most capable participants were able to meaningfully use all categories, with precise thresholds between categories. CONCLUSIONS The authors present a decision-aligned response model that demonstrates the confusability of a particular ECG and the skill with which a clinician can distinguish 2 diagnoses along a continuum of confusability. These results have broad implications for testing and for learning to manage uncertainty in diagnosis.
Collapse
|
7
|
Hatala R, Ginsburg S, Gauthier S, Melvin L, Taylor D, Gingerich A. Supervising the senior medical resident: Entrusting the role, supporting the tasks. MEDICAL EDUCATION 2022; 56:1194-1202. [PMID: 35869566 DOI: 10.1111/medu.14883] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/15/2022] [Revised: 07/09/2022] [Accepted: 07/19/2022] [Indexed: 06/15/2023]
Abstract
INTRODUCTION Postgraduate competency-based medical education has been implemented with programmatic assessment that relies on entrustment-based ratings. Yet, in less procedurally oriented specialties such as internal medicine, the relationship between entrustment and supervision remains unclear. We undertook the current study to address how internal medicine supervisors conceptualise entrusting senior medical residents while supervising them on the acute care wards. METHODS Guided by constructivist grounded theory, we interviewed 19 physicians who regularly supervised senior internal medicine residents on inpatient wards at three Canadian universities. We developed a theoretical model through iterative cycles of data collection and analysis using a constant comparative process. RESULTS On the internal medicine ward, the senior resident role is viewed as a fundamentally managerial and rudimentary version of the supervisor's role. Supervisors come to trust their residents in the senior role through an early 'hands-on' period of assessment followed by a gradual withdrawal of support to promote independence. When considering entrustment, supervisors focused on entrusting a particular scope of the senior resident role as opposed to entrustment of individual tasks. Irrespective of the scope of the role that was entrusted, supervisors at times stepped in and stepped back to support specific tasks. CONCLUSION Supervisors' stepping in and stepping back to support individual tasks on the acute care ward has an inconsistent relationship to their entrustment of the resident with a particular scope of the senior resident role. In this context, entrustment-based assessment would need to capture more of the holistic perspective of the supervisor's entrustment of the senior resident role. Understanding the dance of supervision, from relatively static overall support of the resident in their role, to fluidly stepping in and out for specific patient care tasks, allows us insight into the affordances of the supervisory relationship and how it may be leveraged for assessment.
Collapse
|
8
|
Herman R, Hlivak P, Bernat V, Svetlosak M, Demolder A, Palus T, Urban L, Hatala R. Predictors of long-term atrial fibrillation recurrence after catheter ablation: non-linear analytical approach for individualized prognostic stratification. Eur Heart J 2022. [DOI: 10.1093/eurheartj/ehac544.605] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/14/2022] Open
Abstract
Abstract
Background
Recurrences are common in patients with atrial fibrillation (AF) during long-term follow-up after catheter ablation (CA) for pulmonary vein isolation (PVI). Recently, machine learning (ML) models identifying non-linear relationships among various patient parameters have been applied for prognostic stratification in different cardiac diseases.
Aim
This is a retrospective study aimed to determine whether ML-based models can identify non-linear relationships in individual clinical baseline characteristics and CT-quantified volumetric parameters of epicardial fat tissue (EFT) to aid in prognosing outcome of catheter ablation for PVI in patients with paroxysmal AF.
Methods
A cohort of 92 patients (median age 60.2 [51.9–64.0]; 74% male) with paroxysmal AF (a single persistent AF episode was accepted) undergoing catheter ablation targeting PVI was analysed. All patients underwent cardiac CT imaging and were fitted with implantable loop recorder (ILR) prior to CA. For PVI, radiofrequency CA with electro-anatomical mapping was used in 79 patients, cryoballoon ablation in 13 patients. AF recurrence, defined as AF burden >0.1% after the blanking period (90 days), was continuously assessed by ILR. Feature selection on 23 baseline parameters was performed using random forest models (XGBoostRegressor). Mean absolute Shapley values (|mSHAP| – Shapley Additive expLanations) were used to quantify the relative discriminative power of analysed parameters.
Results
During a follow-up of 3-years, AF recurrence was detected in 58 (63%) patients, 29 (50%) of them underwent a repeat ablation. Five most important predictors of AF recurrence during 3-year follow-up were upper epicardial fat volume, BMI, baseline AF burden, age and pericardial volume (lower segment) (Fig. 1). Upper EFT volume was twice as important for males than females (0.44 and 0.21 |mSHAP| respectively). For a patient with no AF recurrence, low age (41 years) and low upper EFT volume (13.2 ml) were the most important drivers predicting positive ablation outcome (Fig. 2A). In contrast, in a patient with AF recurrence post CA, an above-average EFT volume of 55.5 ml and a high BMI had the most significant net contribution for predicting his failed CA outcome (Fig. 2B).
Conclusion
Non-linear ML analysis applied to our limited cohort of patients with paroxysmal AF undergoing CA suggests: i) a significant association of high EFT volume with ILR determined AF recurrence during a 3-year follow-up; ii) potential role of such analyses for a more granular and highly individualized prediction of outcome of planned CA. However, these results need further testing, and validation in prospective trials.
Funding Acknowledgement
Type of funding sources: None.
Collapse
|
9
|
Herman R, Hlivak P, Bernat V, Svetlosak M, Demolder A, Palus T, Urban L, Hatala R. High epicardial fat volume is associated with atrial fibrillation recurrences after catheter ablation. Eur Heart J 2022. [DOI: 10.1093/eurheartj/ehac544.603] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/12/2022] Open
Abstract
Abstract
Background
Pulmonary vein isolation (PVI) is the cornerstone of catheter ablation therapy for atrial fibrillation (AF). However, even with important technological improvements, AF recurrences are common in long-term follow-up. Recently, epicardial fat tissue (EFT) has been proposed as a potential AF modulator promoting and sustaining atrial arrhythmogenic milieu.
Aim
In this retrospective single-centre study, we sought to evaluate the association between CT-quantified EFT volume and long-term primary PVI outcome assed by means of continuous ECG monitoring with implantable loop recorders (ILR) in paroxysmal AF patients.
Methods
A cohort of 92 patients (median age 60.2 [IQR 51.9–64.0]; 74% male) with paroxysmal AF (a single persistent AF episode was accepted) undergoing catheter ablation targeted at PVI was analysed. Prior to CA all patients underwent cardiac CT imaging with subsequent implantation of implantable loop recorder (ILR). Radiofrequency CA ablation with electro-anatomical mapping for PVI was used in 79 patients, balloon cryoablation was used in 13 patients. Verified complete PVI was defined as procedural endpoint. EFT volume was semi-automatically segmented and quantified on contrast enhanced CT studies. AF recurrence, defined as AF burden >0.1% after the blanking period (90 days), was continuously assessed by ILR. Kaplan-Meier analysis of AF recurrence-free survival at 1-year, 2-year and 3-year follow-up periods compared patients stratified into EFT volume tertiles.
Results
During a median follow-up of 52.6 months [IQR 43–64], AF recurrence was detected in 58 (63%) patients, 29 (50%) of them underwent a repeat ablation. Patients in high, mid, and low total EFT volume tertiles had a median time to late AF recurrence of 154, 373, and 284 days (Fig. 1A). At a 1-year follow-up, patients in the high EFT volume tertile had the worst AF ablation outcome (p=0.023). However, this difference gradually faded-out and became non-significant at 2 and 3 years. ROC analysis revealed an EFT of 95 ml as an optimal cut-off (p=0.002) for expected arrhythmia recurrence post acutely successful PVI (Fig. 1B).
Conclusion
EFT volume quantification in a relatively homogenous, continuously monitored cohort of patients undergoing catheter ablation for paroxysmal AF revealed a significant association between AF recurrence at 1 year post ablation and volume of epicardial fat. These results suggest a possible role of epicardial adipose tissue as an adverse pro-fibrillatory factor. Identified EFT volume cut-offs have to be validated prospectively on a larger and balanced patient population.
Funding Acknowledgement
Type of funding sources: None.
Collapse
|
10
|
Wong SN, Luo CJ, MacDonald G, Hatala R. A qualitative study of medical students' perceptions of resident feedback. MEDICAL EDUCATION 2022; 56:994-1001. [PMID: 35639522 DOI: 10.1111/medu.14847] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/20/2021] [Revised: 05/25/2022] [Accepted: 05/27/2022] [Indexed: 06/15/2023]
Abstract
CONTEXT Residents play a pivotal role in medical students' clinical education. From a feedback lens, the near-peer relationship between student and resident holds the potential to foster an educational alliance that could influence learning. We undertook the current qualitative study to explore medical students' perceptions of feedback experiences with residents, addressing when, how and why (and conversely when not and why not) resident feedback plays a role in their clinical education. METHODS Our methodology was qualitative interpretive description, informed by phenomenology. We conducted 24 semi-structured interviews with third and fourth year medical students at one institution. The interviews aimed to foster rich discussion about students' feedback experiences with residents during clinical rotations. Data collection and analysis proceeded iteratively. Initial interviews were independently open-coded by three investigators and then collaboratively refined. Codes were applied to subsequent interviews, and new codes were developed. During the final stages of analysis, we organised our themes by drawing on a sociocultural perspective to examine students' perceptions of relationship-building with residents and when and how this influenced feedback and learning. RESULTS From the students' perspectives, when residents contributed to building interpersonal relationships with students, this in turn influenced students' receptivity to both encouraging and constructive feedback conversations. In the context of resident-student relationships that were perceived as supportive, resident feedback influenced how students approached learning and working in the clinical environment, as well as students' visions of their future selves. In unsupportive relationships, students were less inclined to engage in feedback with residents and students noted resident behaviours that they wanted to avoid in themselves. CONCLUSION Residents are uniquely positioned to create a strong educational alliance with students in which feedback conversations can flourish. Focusing educational efforts on resident feedback conversations has the potential to significantly impact the feedback culture of our clinical environments.
Collapse
|
11
|
Hatala R, Tavares W. Workplace-based licensing assessments: an idea worth considering? CANADIAN MEDICAL EDUCATION JOURNAL 2022; 13:115-116. [PMID: 36091735 PMCID: PMC9441125 DOI: 10.36834/cmej.73837] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
|
12
|
Spencer M, Sherbino J, Hatala R. Examining the validity argument for the Ottawa Surgical Competency Operating Room Evaluation (OSCORE): a systematic review and narrative synthesis. ADVANCES IN HEALTH SCIENCES EDUCATION : THEORY AND PRACTICE 2022; 27:659-689. [PMID: 35511356 DOI: 10.1007/s10459-022-10114-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/15/2021] [Accepted: 04/02/2022] [Indexed: 06/14/2023]
Abstract
The Ottawa Surgical Competency Operating Room Evaluation (OSCORE) is an assessment tool that has gained prominence in postgraduate competency-based training programs. We undertook a systematic review and narrative synthesis to articulate the underlying validity argument in support of this tool. Although originally developed to assess readiness for independent performance of a procedure, contemporary implementation includes using the OSCORE for entrustment supervision decisions. We used systematic review methodology to search, identify, appraise and abstract relevant articles from 2005 to September 2020, across MEDLINE, EMBASE and Google Scholar databases. Nineteen original, English-language, quantitative or qualitative articles addressing the use of the OSCORE for health professionals' assessment were included. We organized and synthesized the validity evidence according to Kane's framework, articulating the validity argument and identifying evidence gaps. We demonstrate a reasonable validity argument for the OSCORE in surgical specialties, based on assessing surgical competence as readiness for independent performance for a given procedure, which relates to ad hoc, retrospective, entrustment supervision decisions. The scoring, generalization and extrapolation inferences are well-supported. However, there is a notable lack of implications evidence focused on the impact of the OSCORE on summative decision-making within surgical training programs. In non-surgical specialties, the interpretation/use argument for the OSCORE has not been clearly articulated. The OSCORE has been reduced to a single-item global rating scale, and there is limited validity evidence to support its use in workplace-based assessment. Widespread adoption of the OSCORE must be informed by concurrent data collection in more diverse settings and specialties.
Collapse
|
13
|
Tang B, Sandarage R, Chai J, Dawson KA, Dutkiewicz KR, Saad S, Kitchin V, Hatala R, McCormick I, Kassen B. A systematic review of evidence-based practices for clinical education and health care delivery in the clinical teaching unit. CMAJ 2022; 194:E186-E194. [PMID: 35165130 PMCID: PMC8900805 DOI: 10.1503/cmaj.202400] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 11/19/2021] [Indexed: 11/02/2022] Open
Abstract
Background: Methods: Results: Interpretation:
Collapse
|
14
|
Ginsburg S, Watling CJ, Schumacher DJ, Gingerich A, Hatala R. Numbers Encapsulate, Words Elaborate: Toward the Best Use of Comments for Assessment and Feedback on Entrustment Ratings. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2021; 96:S81-S86. [PMID: 34183607 DOI: 10.1097/acm.0000000000004089] [Citation(s) in RCA: 23] [Impact Index Per Article: 7.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
The adoption of entrustment ratings in medical education is based on a seemingly simple premise: to align workplace-based supervision with resident assessment. Yet it has been difficult to operationalize this concept. Entrustment rating forms combine numeric scales with comments and are embedded in a programmatic assessment framework, which encourages the collection of a large quantity of data. The implicit assumption that more is better has led to an untamable volume of data that competency committees must grapple with. In this article, the authors explore the roles of numbers and words on entrustment rating forms, focusing on the intended and optimal use(s) of each, with a focus on the words. They also unpack the problematic issue of dual-purposing words for both assessment and feedback. Words have enormous potential to elaborate, to contextualize, and to instruct; to realize this potential, educators must be crystal clear about their use. The authors set forth a number of possible ways to reconcile these tensions by more explicitly aligning words to purpose. For example, educators could focus written comments solely on assessment; create assessment encounters distinct from feedback encounters; or use different words collected from the same encounter to serve distinct feedback and assessment purposes. Finally, the authors address the tyranny of documentation created by programmatic assessment and urge caution in yielding to the temptation to reduce words to numbers to make them manageable. Instead, they encourage educators to preserve some educational encounters purely for feedback, and to consider that not all words need to become data.
Collapse
|
15
|
Ten Cate O, Balmer DF, Caretta-Weyer H, Hatala R, Hennus MP, West DC. Entrustable Professional Activities and Entrustment Decision Making: A Development and Research Agenda for the Next Decade. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2021; 96:S96-S104. [PMID: 34183610 DOI: 10.1097/acm.0000000000004106] [Citation(s) in RCA: 21] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
To establish a research and development agenda for Entrustable Professional Activities (EPAs) for the coming decade, the authors, all active in this area of investigation, reviewed recent research papers, seeking recommendations for future research. They pooled their knowledge and experience to identify 3 levels of potential research and development: the micro level of learning and teaching; the meso level of institutions, programs, and specialty domains; and the macro level of regional, national, and international dynamics. Within these levels, the authors categorized their recommendations for research and development. The authors identified 14 discrete themes, each including multiple questions or issues for potential exploration, that range from foundational and conceptual to practical. Much research to date has focused on a variety of issues regarding development and early implementation of EPAs. Future research should focus on large-scale implementation of EPAs to support competency-based medical education (CBME) and on its consequences at the 3 levels. In addition, emerging from the implementation phase, the authors call for rigorous studies focusing on conceptual issues. These issues include the nature of entrustment decisions and their relationship with education and learner progress and the use of EPAs across boundaries of training phases, disciplines and professions, including continuing professional development. International studies evaluating the value of EPAs across countries are another important consideration. Future studies should also remain alert for unintended consequences of the use of EPAs. EPAs were conceptualized to support CBME in its endeavor to improve outcomes of education and patient care, prompting creation of this agenda.
Collapse
|
16
|
Gilchrist T, Hatala R, Gingerich A. A collective case study of supervision and competence judgments on the inpatient internal medicine ward. PERSPECTIVES ON MEDICAL EDUCATION 2021; 10:155-162. [PMID: 33492658 PMCID: PMC8187503 DOI: 10.1007/s40037-021-00652-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/08/2020] [Revised: 12/22/2020] [Accepted: 01/07/2021] [Indexed: 05/05/2023]
Abstract
INTRODUCTION Workplace-based assessment in competency-based medical education employs entrustment-supervision scales to suggest trainee competence. However, clinical supervision involves many factors and entrustment decision-making likely reflects more than trainee competence. We do not fully understand how a supervisor's impression of trainee competence is reflected in their provision of clinical support. We must better understand this relationship to know whether documenting level of supervision truly reflects trainee competence. METHODS We undertook a collective case study of supervisor-trainee dyads consisting of attending internal medicine physicians and senior residents working on clinical teaching unit inpatient wards. We conducted field observations of typical daily activities and semi-structured interviews. Data was analysed within each dyad and compared across dyads to identify supervisory behaviours, what triggered the behaviours, and how they related to judgments of trainee competence. RESULTS Ten attending physician-senior resident dyads participated in the study. We identified eight distinct supervisory behaviours. The behaviours were enacted in response to trainee and non-trainee factors. Supervisory behaviours corresponded with varying assessments of trainee competence, even within a dyad. A change in the attending's judgment of the resident's competence did not always correspond with a change in subsequent observable supervisory behaviours. DISCUSSION There was no consistent relationship between a trigger for supervision, the judgment of trainee competence, and subsequent supervisory behaviour. This has direct implications for entrustment assessments tying competence to supervisory behaviours, because supervision is complex. Workplace-based assessments that capture narrative data including the rationale for supervisory behaviours may lead to deeper insights than numeric entrustment ratings.
Collapse
|
17
|
Schmitt J, De Sousa J, Bulava A, Golovchiner G, Hatala R, Anguera I, Reinke F, Wenzel B, Noelker G. Impact of the Covid-19 related lockdown on physical activity, heart rate and arrhythmia burden in a large prospective cohort of CHF patients. Europace 2021. [PMCID: PMC8194656 DOI: 10.1093/europace/euab116.471] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022] Open
Abstract
Funding Acknowledgements Type of funding sources: Private company. Main funding source(s): BIOTRONIK SE & Co. KG OnBehalf BIO|STREAM.HF Background At the beginning of the Covid-19 pandemic in spring 2020, governments around the world issued curfews and other stay at home orders (‘lockdown’) to limit the spread of the SARS-CoV19 virus. This may have forced people to decrease their physical activity. Physical inactivity as well as social stress is known to be especially deleterious for heart failure (HF) patients. The BIO|STREAM.HF study enrolled such HF patients into a prospective registry with Home Monitoring. Purpose We aimed to evaluate the impact of the lockdown during the first Covid-19 pandemic wave on physical activity and arrhythmia burden of heart failure patients. Methods We analysed daily transmitted data of patients enrolled into a large international registry (BIO|STREAM.HF) being implanted with a cardiac resynchronization therapy (CRT) devices. Patients with NYHA ≥ II and LVEF ≤ 40% before CRT implantation were selected. Intra-individual weekly mean and median values were calculated for the following daily transmitted parameters: physical activity (measured as % of the day during which the patient moves), atrial arrhythmia burden, mean heart rate (at rest), PP variability, PVC burden, and rate of biventricular pacing. Values were calculated for 12 weeks before and 12 weeks after the country-specific effective date of most rigorous restrictions in spring 2020 to visualize the general trend of parameter changes. Moreover, values for intra-individual changes between three 28-days periods (before, during, and after the lockdown) were calculated. Results Of 444 patients, 76% were male. They had a mean age of 69 ± 10 years and LVEF of 28.2 ± 6.7%. HF was of ischemic etiology in 42% of cases and they were in NYHA class II (47.5%), III (50.0%) or IV (2.5%). On average, patients were active for 9% of the day (2 h 10 min). The physical activity decreased by approx. 10% with the onset of the lockdown (figure 1) and recovered within the following eight weeks. Comparison of the 28-days periods before, during and after the lockdown showed a statistically significant intra-individual decrease in physical activity (mean decrease 9 min per day) during the lockdown compared to pre- and post-lockdown values and a trend toward reduced mean heart rates. In parallel, a significant increase in device detected atrial arrhythmia burden (mean increase 17 min per day) was observed. All other parameters did not change significantly. Conclusion Our results show that patients reduced their physical activity during the Covid-19 related lockdown in spring 2020. This was associated with an increase in atrial arrhythmia burden and a reduction of the mean heart rate. Prognostic implications of these results will further be analysed.
Abstract Figure. ![]()
Collapse
|
18
|
Howard NM, Cook DA, Hatala R, Pusic MV. Learning Curves in Health Professions Education Simulation Research: A Systematic Review. Simul Healthc 2021; 16:128-135. [PMID: 32675731 DOI: 10.1097/sih.0000000000000477] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/18/2022]
Abstract
SUMMARY STATEMENT Learning curves are used in health professions education to graphically represent paths to competence and expertise. However, research using learning curves often omit important information. The authors conducted a systematic review of the reporting quality of learning curves in simulation-based education research to identify specific areas for improvement. Reviewers extracted information on graphical, statistical, and conceptual elements. The authors identified 230 eligible articles. Most learning curve elements were reported infrequently, including use of an optimal linking function, detailed description of feedback or learning intervention, use of advanced visualization techniques such as overlaying and stacking, and depiction of competency thresholds. Reporting did not improve over time for most elements. Reporting of learning curves in health professions education research is incomplete and often underutilizes their desirable properties. Recommendations for improvement of the statistical, graphical, and conceptual reporting of learning curves, as well as applications to simulation research and education, are presented.
Collapse
|
19
|
Bohm A, Urban L, Tothova L, Bezak B, Uher T, Musil P, Kyselovic J, Lipton J, Olejnik P, Hatala R. Concentration of apelin inversely correlates with atrial fibrillation burden. ACTA ACUST UNITED AC 2021; 122:165-171. [PMID: 33618523 DOI: 10.4149/bll_2021_026] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
AIM Asymptomatic atrial fibrillation (AF) detection and pulmonary veins isolation (PVI) outcome prediction remain challenging. Our aim was to study the association between apelin and paroxysmal AF in patients undergoing radiofrequency catheter PVI. METHODS Sixty-three consecutive patients (55 ± 8years, 12 females) with paroxysmal AF without a structural heart disease and implanted ECG loop recorders undergoing PVI and healthy control group of 34 persons (41 ± 9.5years, 21 females) were included. Apelin plasmatic concentrations were measured before and three months after PVI. AF burden was continually assessed for three years. RESULTS Apelin was significantly decreased in AF patients compared to the healthy controls (0.79 ± 0.09 vs 0.98 ± 0.06 ng/ml; p < 0.00001). Apelin plasmatic concentration of 0.89 ng/ml had 94 % specificity and 89 % sensitivity for AF prediction with the area under the curve (AUC) of 0.96. After propensity matching to sex, age and comorbidities, apelin concentration was significantly lower in AF group (0.78 ± 0.1 vs 0.99 ±0.06 ng/ml; p < 0.0001; AUC: 0.97). There was a significant inverse correlation between apelin concentration and AF burden both before and after PVI (Rho = ‒0.22; p = 0.05) and (Rho = ‒0.51; p = 0.006), respectively. There was no significant association between pre-PVI apelin and PVI long-term outcome. CONCLUSION In patients without a structural heart disease apelin showed a significant specificity and sensitivity for AF prediction and inversely correlated with AF burden (Tab. 3, Fig. 3, Ref. 34).
Collapse
|
20
|
Darras KE, Forster BB, Spouge R, de Bruin ABH, Arnold A, Nicolaou S, Hu J, Hatala R, van Merriënboer J. Virtual Dissection with Clinical Radiology Cases Provides Educational Value to First Year Medical Students. Acad Radiol 2020; 27:1633-1640. [PMID: 31786075 DOI: 10.1016/j.acra.2019.09.031] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/07/2019] [Revised: 09/15/2019] [Accepted: 09/17/2019] [Indexed: 10/25/2022]
Abstract
RATIONALE AND AIM In virtual dissection, three-dimensional computed tomography scans are viewed on a near-life size virtual dissection table and through touchscreen technology, students work together to manipulate the data to perform their dissection. The purpose of this study was to develop a Virtual Dissection Curriculum for first year medical students and to assess its educational value as well as students' preferred pedagogy for learning with this new technology. METHODS One hundred and five first-year medical students participated in a case-based virtual dissection curriculum and were invited to complete a theory-based post experience survey. Eight unique clinical cases were selected based on the first-year curricular objectives and divided into four 30-minute sessions. In groups of 6-8, students reviewed the cases with a radiologist. First, students' reactions to virtual dissection were measured by three constructs using a 5-point Likert scale: quality of curriculum design (11 questions), impact on learning (7 questions), and comfort with technology (3 questions). Second, students ranked the usefulness of six pedagogical approaches for this technology. Responses were tabulated and rank order item lists were generated statistically using the Schulze method where appropriate. RESULTS The survey response rate was 83% (87/105). Overall, students' reactions to virtual dissection were positive across all three measured constructs. Most students indicated that the cases were of an appropriate level of difficulty (90%) and that virtual dissection improved their understanding of disease and pathology (89%), the clinical relevance of anatomy (77%), and visuospatial relationships (64%). Almost all students (94%) reported that the curriculum improved understanding of the role of the radiologist in patient care. Students felt that the "very useful" pedagogical approaches were small group demonstration (68%) and problem-based learning (51%). CONCLUSION First-year medical students perceive the use of virtual dissection as a valuable tool for learning anatomy and radiology. This technology enables the integration of clinical cases and radiology content into preclinical learning.
Collapse
|
21
|
Uher T, Bohm A, Urban L, Tothova L, Bacharova L, Musil P, Kyselovic J, Michalek P, Vachalcova M, Olejnik P, Hatala R. Association of apelin and AF in patients with implanted loop recorders undergoing catheter ablation. BRATISL MED J 2020; 121:484-487. [PMID: 32990001 DOI: 10.4149/bll_2020_079] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
BACKGROUND Previous studies showed an association between apelin and atrial fibrillation (AF). The aim of this study was to analyse the effect of pulmonary vein isolation (PVI) in patients with paroxysmal AF on plasmatic apelin concentrations. METHODS Nine consecutive patients (aged from 43 to 69 years, 3 females and 6 males) with documented paroxysmal atrial fibrillation and implanted loop recorders (ILR) for continuous ECG monitoring were included in this study. All the patients underwent a radiofrequency catheter ablation with PVI. RESULTS The plasmatic concentration of apelin increased after PVI. The average plasmatic concentration of apelin before PVI was 0.299 ng/ml (±0.16), 3 months after PVI 0.462 ng/ml (±0.10) and 9 months after PVI 0.565 ng/ml (±0.146). There was an increase in the concentration of apelin 3 months and 9 months after the PVI by 0.163 ng/ml (p=0.07) and by 0.266 ng/ml (p=0.01), respectively. The concentration of apelin inversely correlated with the AF burden (r=-0.44, p=0.03). CONCLUSIONS Our study showed a significant increase in apelin levels after the reduction of AF burden via PVI and an inverse correlation with AF burden. Apelin might be a promising marker of AF (Tab. 2, Fig. 2, Ref. 28).
Collapse
|
22
|
Hatala R. Proposal for complementary targeted public health anti-epidemic measures during the pandemics of COVID-19. BRATISL MED J 2020; 121:459-461. [PMID: 32989995 DOI: 10.4149/bll_2020_76] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
|
23
|
MacNeil K, Cuncic C, Voyer S, Butler D, Hatala R. Necessary but not sufficient: identifying conditions for effective feedback during internal medicine residents' clinical education. ADVANCES IN HEALTH SCIENCES EDUCATION : THEORY AND PRACTICE 2020; 25:641-654. [PMID: 31872326 DOI: 10.1007/s10459-019-09948-8] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/06/2019] [Accepted: 12/12/2019] [Indexed: 06/10/2023]
Abstract
Competency-based medical education and programmatic assessment intend to increase the opportunities for meaningful feedback, yet these conversations remain elusive. By comparing resident and faculty perceptions of feedback opportunities within one internal medicine residency training program, we sought to understand whether and how principles underlying meaningful feedback could be supported or constrained across a variety of feedback opportunities. Using case-study qualitative methodology, interviews and focus groups were conducted to explore 19 internal medicine residents' and 7 faculty members' perceptions of feedback across a variety of feedback opportunities: coaching, mini-CEXs, in-training evaluation reports and routine clinical supervision. Our data analysis moved iteratively between developing conceptual understandings and fine-grained analyses, while attending to both deductive and inductive analysis. Our results suggest that all feedback opportunities, including those created through formalized assessments, can foster meaningful feedback if faculty establish a trusting relationship with the resident, base their feedback on direct observation and support resident learning. However, formalized assessments were often perceived as inhibiting the conditions for meaningful feedback. A coaching program provided a context in which meaningful feedback could arise, in part because faculty were supported in shifting their focus from patient to resident. Meaningful feedback in clinical education may be fostered across a variety of feedback opportunities, however, it is often constrained by assessment. We must consider whether increasing the frequency of formative assessments may inhibit efforts to improve our feedback cultures while, in contrast, freeing up faculty to focus on supporting resident learning could improve these cultures.
Collapse
|
24
|
Shariff F, Hatala R, Regehr G. The nature of learning from simulation: Now I know it, now I'll do it, I'll work on that. MEDICAL EDUCATION 2020; 54:652-659. [PMID: 32162379 DOI: 10.1111/medu.14153] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/18/2019] [Revised: 03/05/2020] [Accepted: 03/09/2020] [Indexed: 06/10/2023]
Abstract
CONTEXT Ongoing learning in complex clinical environments requires health professionals to assess their own performance, manage their learning, and modify their practices based on self-monitored progress. Self-regulated learning (SRL) theory suggests that although learners may be capable of such learning, they often need guidance to enact it effectively. Debriefings following simulation may be an ideal time to support learners' use of SRL in targeted areas, but the extent to which they are optimally fostering these practices has not been examined. METHODS A qualitative study informed by grounded theory methodology was conducted in the context of three interprofessional in situ trauma simulations at our level 1 trauma centre. A total of 18 participants were interviewed both immediately and 5-6 weeks after the simulation experience. Transcripts were analysed using an iterative constant comparative approach to explore concepts and themes regarding the nature of learning from and after simulation. RESULTS During initial interviews, there were many examples of acquired content knowledge and straightforward practice changes that might not require ongoing SRL to enact well in practice. However, even for skills identified as needing to be 'worked on,' SRL strategies were lacking. At follow-up interviews, some participants had evolved more specific learning goals and rudimentary plans for implementation and improvement, but suggested this was prompted by the study interview questions rather than the simulation debriefing itself. CONCLUSIONS Overall, participants did not engage in fulsome development of SRL plans based on the simulation and debriefing; however, there were elements of SRL present, particularly after participants were given time to reflect on the interview questions and their own goals. This suggests that simulation training can support the use of SRL. However, debriefing approaches might be better optimised to take full advantage of the opportunity to encourage and foster SRL in practice after the simulation is over.
Collapse
|
25
|
Shariff F, Hatala R, Regehr G. Learning After the Simulation Is Over: The Role of Simulation in Supporting Ongoing Self-Regulated Learning in Practice. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2020; 95:523-526. [PMID: 31725461 DOI: 10.1097/acm.0000000000003078] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
The complex and dynamic nature of the clinical environment often requires health professionals to assess their own performance, manage their learning, and modify their practices based on self-monitored progress. Self-regulated learning studies suggest that while learners may be capable of such in situ learning, they often need guidance to enact it effectively. In this Perspective, the authors argue that simulation training may be an ideal venue to prepare learners for self-regulated learning in the clinical setting but may not currently be optimally fostering self-regulated learning practices. They point out that current simulation debriefing models emphasize the need to synthesize a set of identified goals for practice change (what behaviors might be modified) but do not address how learners might self-monitor the success of their implementation efforts and modify their learning plans based on this monitoring when back in the clinical setting. The authors describe the current models of simulation-based learning implied in the simulation literature and suggest potential targets in the simulation training process, which might be optimized to allow medical educators to take full advantage of the opportunity simulation provides to support and promote ongoing self-regulated learning in practice.
Collapse
|