Dual purposes by design: exploring alignment between residents' and academic advisors' documents in a longitudinal program

Longitudinal academic advising (AA) and coaching programs are increasingly implemented in competency based medical education (CBME) to help residents reflect and act on the voluminous assessment data they receive. Documents created by residents for purposes of reflection are often used for a second, summative purpose-to help competence committees make decisions-which may be problematic. Using inductive, thematic analysis we analyzed written comments generated by 21 resident-AA dyads in one large internal medicine program who met over a 2 year period to determine what residents write when asked to reflect, how this aligns with what the AAs report, and what changes occur over time (total 109 resident self-reflections and 105 AA reports). Residents commented more on their developing autonomy, progress and improvement than AAs, who commented far more on performance measures. Over time, residents' writing shifted away from intrinsic roles, patient care and improvement towards what AAs focused on, including getting EPAs (entrustable professional activities), studying and exams. For EPAs, the emphasis was on getting sufficient numbers rather than reflecting on what residents were learning. Our findings challenge the practice of dual-purposing documents, by questioning the blurring of formative and summative intent, the structure of forms and their multiple conflicting purposes, and assumptions about the advising relationship over time. Our study suggests a need to re-evaluate how reflective documents are used in CBME programs. Further research should explore whether and how documentation can best be used to support resident growth and development.

Goal co-construction and dialogue in an internal medicine longitudinal coaching programme

BACKGROUND

Longitudinal coaching in residency programmes is becoming commonplace and requires iterative and collaborative discussions between coach and resident, with the shared development of goals. However, little is known about how goal development unfolds within coaching conversations over time and the effects these conversations have. We therefore built on current coaching theory by analysing goal development dialogues within resident and faculty coaching relationships.

METHODS

This was a qualitative study using interpretive description methodology. Eight internal medicine coach-resident dyads consented to audiotaping coaching meetings over a 1-year period. Transcripts from meetings and individual exit interviews were analysed thematically using goal co-construction as a sensitising concept.

RESULTS

Two themes were developed: (i) The content of goals discussed in coaching meetings focused on how to be a resident, with little discussion around challenges in direct patient care, and (ii) co-construction mainly occurred in how to meet goals, rather than in prioritising goals or co-constructing new goals.

CONCLUSIONS

In analysing goal development in the coach-resident relationships, conversations focused mainly around how to manage as a resident rather than how to improve direct patient care. This may be because academic coaching provides space separate from clinical work to focus on the stage-specific professional identity development of a resident. Going forward, focus should be on how to optimise longitudinal coaching conversations to ensure co-regulation and reflection on both clinical competencies and professional identity formation.

Modeling Diagnostic Expertise in Cases of Irreducible Uncertainty: The Decision-Aligned Response Model

PURPOSE

Assessing expertise using psychometric models usually yields a measure of ability that is difficult to generalize to the complexity of diagnoses in clinical practice. However, using an item response modeling framework, it is possible to create a decision-aligned response model that captures a clinician's decision-making behavior on a continuous scale that fully represents competing diagnostic possibilities. In this proof-of-concept study, the authors demonstrate the necessary statistical conceptualization of this model using a specific electrocardiogram (ECG) example.

METHOD

The authors collected a range of ECGs with elevated ST segments due to either ST-elevation myocardial infarction (STEMI) or pericarditis. Based on pilot data, 20 ECGs were chosen to represent a continuum from "definitely STEMI" to "definitely pericarditis," including intermediate cases in which the diagnosis was intentionally unclear. Emergency medicine and cardiology physicians rated these ECGs on a 5-point scale ("definitely STEMI" to "definitely pericarditis"). The authors analyzed these ratings using a graded response model showing the degree to which each participant could separate the ECGs along the diagnostic continuum. The authors compared these metrics with the discharge diagnoses noted on chart review.

RESULTS

Thirty-seven participants rated the ECGs. As desired, the ECGs represented a range of phenotypes, including cases where participants were uncertain in their diagnosis. The response model showed that participants varied both in their propensity to diagnose one condition over another and in where they placed the thresholds between the 5 diagnostic categories. The most capable participants were able to meaningfully use all categories, with precise thresholds between categories.

CONCLUSIONS

The authors present a decision-aligned response model that demonstrates the confusability of a particular ECG and the skill with which a clinician can distinguish 2 diagnoses along a continuum of confusability. These results have broad implications for testing and for learning to manage uncertainty in diagnosis.

Supervising the senior medical resident: Entrusting the role, supporting the tasks

INTRODUCTION

Postgraduate competency-based medical education has been implemented with programmatic assessment that relies on entrustment-based ratings. Yet, in less procedurally oriented specialties such as internal medicine, the relationship between entrustment and supervision remains unclear. We undertook the current study to address how internal medicine supervisors conceptualise entrusting senior medical residents while supervising them on the acute care wards.

METHODS

Guided by constructivist grounded theory, we interviewed 19 physicians who regularly supervised senior internal medicine residents on inpatient wards at three Canadian universities. We developed a theoretical model through iterative cycles of data collection and analysis using a constant comparative process.

RESULTS

On the internal medicine ward, the senior resident role is viewed as a fundamentally managerial and rudimentary version of the supervisor's role. Supervisors come to trust their residents in the senior role through an early 'hands-on' period of assessment followed by a gradual withdrawal of support to promote independence. When considering entrustment, supervisors focused on entrusting a particular scope of the senior resident role as opposed to entrustment of individual tasks. Irrespective of the scope of the role that was entrusted, supervisors at times stepped in and stepped back to support specific tasks.

CONCLUSION

Supervisors' stepping in and stepping back to support individual tasks on the acute care ward has an inconsistent relationship to their entrustment of the resident with a particular scope of the senior resident role. In this context, entrustment-based assessment would need to capture more of the holistic perspective of the supervisor's entrustment of the senior resident role. Understanding the dance of supervision, from relatively static overall support of the resident in their role, to fluidly stepping in and out for specific patient care tasks, allows us insight into the affordances of the supervisory relationship and how it may be leveraged for assessment.

Predictors of long-term atrial fibrillation recurrence after catheter ablation: non-linear analytical approach for individualized prognostic stratification

Background

Recurrences are common in patients with atrial fibrillation (AF) during long-term follow-up after catheter ablation (CA) for pulmonary vein isolation (PVI). Recently, machine learning (ML) models identifying non-linear relationships among various patient parameters have been applied for prognostic stratification in different cardiac diseases.

Aim

This is a retrospective study aimed to determine whether ML-based models can identify non-linear relationships in individual clinical baseline characteristics and CT-quantified volumetric parameters of epicardial fat tissue (EFT) to aid in prognosing outcome of catheter ablation for PVI in patients with paroxysmal AF.

Methods

A cohort of 92 patients (median age 60.2 [51.9–64.0]; 74% male) with paroxysmal AF (a single persistent AF episode was accepted) undergoing catheter ablation targeting PVI was analysed. All patients underwent cardiac CT imaging and were fitted with implantable loop recorder (ILR) prior to CA. For PVI, radiofrequency CA with electro-anatomical mapping was used in 79 patients, cryoballoon ablation in 13 patients. AF recurrence, defined as AF burden >0.1% after the blanking period (90 days), was continuously assessed by ILR. Feature selection on 23 baseline parameters was performed using random forest models (XGBoostRegressor). Mean absolute Shapley values (|mSHAP| – Shapley Additive expLanations) were used to quantify the relative discriminative power of analysed parameters.

Results

During a follow-up of 3-years, AF recurrence was detected in 58 (63%) patients, 29 (50%) of them underwent a repeat ablation. Five most important predictors of AF recurrence during 3-year follow-up were upper epicardial fat volume, BMI, baseline AF burden, age and pericardial volume (lower segment) (Fig. 1). Upper EFT volume was twice as important for males than females (0.44 and 0.21 |mSHAP| respectively). For a patient with no AF recurrence, low age (41 years) and low upper EFT volume (13.2 ml) were the most important drivers predicting positive ablation outcome (Fig. 2A). In contrast, in a patient with AF recurrence post CA, an above-average EFT volume of 55.5 ml and a high BMI had the most significant net contribution for predicting his failed CA outcome (Fig. 2B).

Conclusion

Non-linear ML analysis applied to our limited cohort of patients with paroxysmal AF undergoing CA suggests: i) a significant association of high EFT volume with ILR determined AF recurrence during a 3-year follow-up; ii) potential role of such analyses for a more granular and highly individualized prediction of outcome of planned CA. However, these results need further testing, and validation in prospective trials.

Funding Acknowledgement

Type of funding sources: None.

High epicardial fat volume is associated with atrial fibrillation recurrences after catheter ablation

Background

Pulmonary vein isolation (PVI) is the cornerstone of catheter ablation therapy for atrial fibrillation (AF). However, even with important technological improvements, AF recurrences are common in long-term follow-up. Recently, epicardial fat tissue (EFT) has been proposed as a potential AF modulator promoting and sustaining atrial arrhythmogenic milieu.

Aim

In this retrospective single-centre study, we sought to evaluate the association between CT-quantified EFT volume and long-term primary PVI outcome assed by means of continuous ECG monitoring with implantable loop recorders (ILR) in paroxysmal AF patients.

Methods

A cohort of 92 patients (median age 60.2 [IQR 51.9–64.0]; 74% male) with paroxysmal AF (a single persistent AF episode was accepted) undergoing catheter ablation targeted at PVI was analysed. Prior to CA all patients underwent cardiac CT imaging with subsequent implantation of implantable loop recorder (ILR). Radiofrequency CA ablation with electro-anatomical mapping for PVI was used in 79 patients, balloon cryoablation was used in 13 patients. Verified complete PVI was defined as procedural endpoint. EFT volume was semi-automatically segmented and quantified on contrast enhanced CT studies. AF recurrence, defined as AF burden >0.1% after the blanking period (90 days), was continuously assessed by ILR. Kaplan-Meier analysis of AF recurrence-free survival at 1-year, 2-year and 3-year follow-up periods compared patients stratified into EFT volume tertiles.

Results

During a median follow-up of 52.6 months [IQR 43–64], AF recurrence was detected in 58 (63%) patients, 29 (50%) of them underwent a repeat ablation. Patients in high, mid, and low total EFT volume tertiles had a median time to late AF recurrence of 154, 373, and 284 days (Fig. 1A). At a 1-year follow-up, patients in the high EFT volume tertile had the worst AF ablation outcome (p=0.023). However, this difference gradually faded-out and became non-significant at 2 and 3 years. ROC analysis revealed an EFT of 95 ml as an optimal cut-off (p=0.002) for expected arrhythmia recurrence post acutely successful PVI (Fig. 1B).

Conclusion

EFT volume quantification in a relatively homogenous, continuously monitored cohort of patients undergoing catheter ablation for paroxysmal AF revealed a significant association between AF recurrence at 1 year post ablation and volume of epicardial fat. These results suggest a possible role of epicardial adipose tissue as an adverse pro-fibrillatory factor. Identified EFT volume cut-offs have to be validated prospectively on a larger and balanced patient population.

Funding Acknowledgement

Type of funding sources: None.

A qualitative study of medical students' perceptions of resident feedback

CONTEXT

Residents play a pivotal role in medical students' clinical education. From a feedback lens, the near-peer relationship between student and resident holds the potential to foster an educational alliance that could influence learning. We undertook the current qualitative study to explore medical students' perceptions of feedback experiences with residents, addressing when, how and why (and conversely when not and why not) resident feedback plays a role in their clinical education.

METHODS

Our methodology was qualitative interpretive description, informed by phenomenology. We conducted 24 semi-structured interviews with third and fourth year medical students at one institution. The interviews aimed to foster rich discussion about students' feedback experiences with residents during clinical rotations. Data collection and analysis proceeded iteratively. Initial interviews were independently open-coded by three investigators and then collaboratively refined. Codes were applied to subsequent interviews, and new codes were developed. During the final stages of analysis, we organised our themes by drawing on a sociocultural perspective to examine students' perceptions of relationship-building with residents and when and how this influenced feedback and learning.

RESULTS

From the students' perspectives, when residents contributed to building interpersonal relationships with students, this in turn influenced students' receptivity to both encouraging and constructive feedback conversations. In the context of resident-student relationships that were perceived as supportive, resident feedback influenced how students approached learning and working in the clinical environment, as well as students' visions of their future selves. In unsupportive relationships, students were less inclined to engage in feedback with residents and students noted resident behaviours that they wanted to avoid in themselves.

CONCLUSION

Residents are uniquely positioned to create a strong educational alliance with students in which feedback conversations can flourish. Focusing educational efforts on resident feedback conversations has the potential to significantly impact the feedback culture of our clinical environments.

Examining the validity argument for the Ottawa Surgical Competency Operating Room Evaluation (OSCORE): a systematic review and narrative synthesis

The Ottawa Surgical Competency Operating Room Evaluation (OSCORE) is an assessment tool that has gained prominence in postgraduate competency-based training programs. We undertook a systematic review and narrative synthesis to articulate the underlying validity argument in support of this tool. Although originally developed to assess readiness for independent performance of a procedure, contemporary implementation includes using the OSCORE for entrustment supervision decisions. We used systematic review methodology to search, identify, appraise and abstract relevant articles from 2005 to September 2020, across MEDLINE, EMBASE and Google Scholar databases. Nineteen original, English-language, quantitative or qualitative articles addressing the use of the OSCORE for health professionals' assessment were included. We organized and synthesized the validity evidence according to Kane's framework, articulating the validity argument and identifying evidence gaps. We demonstrate a reasonable validity argument for the OSCORE in surgical specialties, based on assessing surgical competence as readiness for independent performance for a given procedure, which relates to ad hoc, retrospective, entrustment supervision decisions. The scoring, generalization and extrapolation inferences are well-supported. However, there is a notable lack of implications evidence focused on the impact of the OSCORE on summative decision-making within surgical training programs. In non-surgical specialties, the interpretation/use argument for the OSCORE has not been clearly articulated. The OSCORE has been reduced to a single-item global rating scale, and there is limited validity evidence to support its use in workplace-based assessment. Widespread adoption of the OSCORE must be informed by concurrent data collection in more diverse settings and specialties.

A systematic review of evidence-based practices for clinical education and health care delivery in the clinical teaching unit

Background:

Methods:

Results:

Interpretation:

Numbers Encapsulate, Words Elaborate: Toward the Best Use of Comments for Assessment and Feedback on Entrustment Ratings

The adoption of entrustment ratings in medical education is based on a seemingly simple premise: to align workplace-based supervision with resident assessment. Yet it has been difficult to operationalize this concept. Entrustment rating forms combine numeric scales with comments and are embedded in a programmatic assessment framework, which encourages the collection of a large quantity of data. The implicit assumption that more is better has led to an untamable volume of data that competency committees must grapple with. In this article, the authors explore the roles of numbers and words on entrustment rating forms, focusing on the intended and optimal use(s) of each, with a focus on the words. They also unpack the problematic issue of dual-purposing words for both assessment and feedback. Words have enormous potential to elaborate, to contextualize, and to instruct; to realize this potential, educators must be crystal clear about their use. The authors set forth a number of possible ways to reconcile these tensions by more explicitly aligning words to purpose. For example, educators could focus written comments solely on assessment; create assessment encounters distinct from feedback encounters; or use different words collected from the same encounter to serve distinct feedback and assessment purposes. Finally, the authors address the tyranny of documentation created by programmatic assessment and urge caution in yielding to the temptation to reduce words to numbers to make them manageable. Instead, they encourage educators to preserve some educational encounters purely for feedback, and to consider that not all words need to become data.

Entrustable Professional Activities and Entrustment Decision Making: A Development and Research Agenda for the Next Decade

To establish a research and development agenda for Entrustable Professional Activities (EPAs) for the coming decade, the authors, all active in this area of investigation, reviewed recent research papers, seeking recommendations for future research. They pooled their knowledge and experience to identify 3 levels of potential research and development: the micro level of learning and teaching; the meso level of institutions, programs, and specialty domains; and the macro level of regional, national, and international dynamics. Within these levels, the authors categorized their recommendations for research and development. The authors identified 14 discrete themes, each including multiple questions or issues for potential exploration, that range from foundational and conceptual to practical. Much research to date has focused on a variety of issues regarding development and early implementation of EPAs. Future research should focus on large-scale implementation of EPAs to support competency-based medical education (CBME) and on its consequences at the 3 levels. In addition, emerging from the implementation phase, the authors call for rigorous studies focusing on conceptual issues. These issues include the nature of entrustment decisions and their relationship with education and learner progress and the use of EPAs across boundaries of training phases, disciplines and professions, including continuing professional development. International studies evaluating the value of EPAs across countries are another important consideration. Future studies should also remain alert for unintended consequences of the use of EPAs. EPAs were conceptualized to support CBME in its endeavor to improve outcomes of education and patient care, prompting creation of this agenda.

A collective case study of supervision and competence judgments on the inpatient internal medicine ward

INTRODUCTION

Workplace-based assessment in competency-based medical education employs entrustment-supervision scales to suggest trainee competence. However, clinical supervision involves many factors and entrustment decision-making likely reflects more than trainee competence. We do not fully understand how a supervisor's impression of trainee competence is reflected in their provision of clinical support. We must better understand this relationship to know whether documenting level of supervision truly reflects trainee competence.

METHODS

We undertook a collective case study of supervisor-trainee dyads consisting of attending internal medicine physicians and senior residents working on clinical teaching unit inpatient wards. We conducted field observations of typical daily activities and semi-structured interviews. Data was analysed within each dyad and compared across dyads to identify supervisory behaviours, what triggered the behaviours, and how they related to judgments of trainee competence.

RESULTS

Ten attending physician-senior resident dyads participated in the study. We identified eight distinct supervisory behaviours. The behaviours were enacted in response to trainee and non-trainee factors. Supervisory behaviours corresponded with varying assessments of trainee competence, even within a dyad. A change in the attending's judgment of the resident's competence did not always correspond with a change in subsequent observable supervisory behaviours.

DISCUSSION

There was no consistent relationship between a trigger for supervision, the judgment of trainee competence, and subsequent supervisory behaviour. This has direct implications for entrustment assessments tying competence to supervisory behaviours, because supervision is complex. Workplace-based assessments that capture narrative data including the rationale for supervisory behaviours may lead to deeper insights than numeric entrustment ratings.

Impact of the Covid-19 related lockdown on physical activity, heart rate and arrhythmia burden in a large prospective cohort of CHF patients

Funding Acknowledgements

Type of funding sources: Private company. Main funding source(s): BIOTRONIK SE & Co. KG

OnBehalf

BIO|STREAM.HF

Background

At the beginning of the Covid-19 pandemic in spring 2020, governments around the world issued curfews and other stay at home orders (‘lockdown’) to limit the spread of the SARS-CoV19 virus. This may have forced people to decrease their physical activity. Physical inactivity as well as social stress is known to be especially deleterious for heart failure (HF) patients. The BIO|STREAM.HF study enrolled such HF patients into a prospective registry with Home Monitoring.

Purpose

We aimed to evaluate the impact of the lockdown during the first Covid-19 pandemic wave on physical activity and arrhythmia burden of heart failure patients.

Methods

We analysed daily transmitted data of patients enrolled into a large international registry (BIO|STREAM.HF) being implanted with a cardiac resynchronization therapy (CRT) devices. Patients with NYHA ≥ II and LVEF ≤ 40% before CRT implantation were selected.

Intra-individual weekly mean and median values were calculated for the following daily transmitted parameters: physical activity (measured as % of the day during which the patient moves), atrial arrhythmia burden, mean heart rate (at rest), PP variability, PVC burden, and rate of biventricular pacing. Values were calculated for 12 weeks before and 12 weeks after the country-specific effective date of most rigorous restrictions in spring 2020 to visualize the general trend of parameter changes. Moreover, values for intra-individual changes between three 28-days periods (before, during, and after the lockdown) were calculated.

Results

Of 444 patients, 76% were male. They had a mean age of 69 ± 10 years and LVEF of 28.2 ± 6.7%. HF was of ischemic etiology in 42% of cases and they were in NYHA class II (47.5%), III (50.0%) or IV (2.5%).

On average, patients were active for 9% of the day (2 h 10 min). The physical activity decreased by approx. 10% with the onset of the lockdown (figure 1) and recovered within the following eight weeks.

Comparison of the 28-days periods before, during and after the lockdown showed a statistically significant intra-individual decrease in physical activity (mean decrease 9 min per day) during the lockdown compared to pre- and post-lockdown values and a trend toward reduced mean heart rates. In parallel, a significant increase in device detected atrial arrhythmia burden (mean increase 17 min per day) was observed. All other parameters did not change significantly.

Conclusion 

Our results show that patients reduced their physical activity during the Covid-19 related lockdown in spring 2020. This was associated with an increase in atrial arrhythmia burden and a reduction of the mean heart rate. Prognostic implications of these results will further be analysed.

Learning Curves in Health Professions Education Simulation Research: A Systematic Review

SUMMARY STATEMENT

Learning curves are used in health professions education to graphically represent paths to competence and expertise. However, research using learning curves often omit important information. The authors conducted a systematic review of the reporting quality of learning curves in simulation-based education research to identify specific areas for improvement. Reviewers extracted information on graphical, statistical, and conceptual elements. The authors identified 230 eligible articles. Most learning curve elements were reported infrequently, including use of an optimal linking function, detailed description of feedback or learning intervention, use of advanced visualization techniques such as overlaying and stacking, and depiction of competency thresholds. Reporting did not improve over time for most elements. Reporting of learning curves in health professions education research is incomplete and often underutilizes their desirable properties. Recommendations for improvement of the statistical, graphical, and conceptual reporting of learning curves, as well as applications to simulation research and education, are presented.

Concentration of apelin inversely correlates with atrial fibrillation burden

AIM

Asymptomatic atrial fibrillation (AF) detection and pulmonary veins isolation (PVI) outcome prediction remain challenging. Our aim was to study the association between apelin and paroxysmal AF in patients undergoing radiofrequency catheter PVI.

METHODS

Sixty-three consecutive patients (55 ± 8years, 12 females) with paroxysmal AF without a structural heart disease and implanted ECG loop recorders undergoing PVI and healthy control group of 34 persons (41 ± 9.5years, 21 females) were included. Apelin plasmatic concentrations were measured before and three months after PVI. AF burden was continually assessed for three years.

RESULTS

Apelin was significantly decreased in AF patients compared to the healthy controls (0.79 ± 0.09 vs 0.98 ± 0.06 ng/ml; p < 0.00001). Apelin plasmatic concentration of 0.89 ng/ml had 94 % specificity and 89 % sensitivity for AF prediction with the area under the curve (AUC) of 0.96. After propensity matching to sex, age and comorbidities, apelin concentration was significantly lower in AF group (0.78 ± 0.1 vs 0.99 ±0.06  ng/ml; p < 0.0001; AUC: 0.97). There was a significant inverse correlation between apelin concentration and AF burden both before and after PVI (Rho = ‒0.22; p = 0.05) and (Rho = ‒0.51; p = 0.006), respectively. There was no significant association between pre-PVI apelin and PVI long-term outcome.

CONCLUSION

In patients without a structural heart disease apelin showed a significant specificity and sensitivity for AF prediction and inversely correlated with AF burden (Tab. 3, Fig. 3, Ref. 34).

Virtual Dissection with Clinical Radiology Cases Provides Educational Value to First Year Medical Students

RATIONALE AND AIM

In virtual dissection, three-dimensional computed tomography scans are viewed on a near-life size virtual dissection table and through touchscreen technology, students work together to manipulate the data to perform their dissection. The purpose of this study was to develop a Virtual Dissection Curriculum for first year medical students and to assess its educational value as well as students' preferred pedagogy for learning with this new technology.

METHODS

One hundred and five first-year medical students participated in a case-based virtual dissection curriculum and were invited to complete a theory-based post experience survey. Eight unique clinical cases were selected based on the first-year curricular objectives and divided into four 30-minute sessions. In groups of 6-8, students reviewed the cases with a radiologist. First, students' reactions to virtual dissection were measured by three constructs using a 5-point Likert scale: quality of curriculum design (11 questions), impact on learning (7 questions), and comfort with technology (3 questions). Second, students ranked the usefulness of six pedagogical approaches for this technology. Responses were tabulated and rank order item lists were generated statistically using the Schulze method where appropriate.

RESULTS

The survey response rate was 83% (87/105). Overall, students' reactions to virtual dissection were positive across all three measured constructs. Most students indicated that the cases were of an appropriate level of difficulty (90%) and that virtual dissection improved their understanding of disease and pathology (89%), the clinical relevance of anatomy (77%), and visuospatial relationships (64%). Almost all students (94%) reported that the curriculum improved understanding of the role of the radiologist in patient care. Students felt that the "very useful" pedagogical approaches were small group demonstration (68%) and problem-based learning (51%).

CONCLUSION

First-year medical students perceive the use of virtual dissection as a valuable tool for learning anatomy and radiology. This technology enables the integration of clinical cases and radiology content into preclinical learning.

Association of apelin and AF in patients with implanted loop recorders undergoing catheter ablation

BACKGROUND

Previous studies showed an association between apelin and atrial fibrillation (AF). The aim of this study was to analyse the effect of pulmonary vein isolation (PVI) in patients with paroxysmal AF on plasmatic apelin concentrations.

METHODS

Nine consecutive patients (aged from 43 to 69 years, 3 females and 6 males) with documented paroxysmal atrial fibrillation and implanted loop recorders (ILR) for continuous ECG monitoring were included in this study. All the patients underwent a radiofrequency catheter ablation with PVI.

RESULTS

The plasmatic concentration of apelin increased after PVI. The average plasmatic concentration of apelin before PVI was 0.299 ng/ml (±0.16), 3 months after PVI 0.462 ng/ml (±0.10) and 9 months after PVI 0.565 ng/ml (±0.146). There was an increase in the concentration of apelin 3 months and 9 months after the PVI by 0.163 ng/ml (p=0.07) and by 0.266 ng/ml (p=0.01), respectively. The concentration of apelin inversely correlated with the AF burden (r=-0.44, p=0.03).

CONCLUSIONS

Our study showed a significant increase in apelin levels after the reduction of AF burden via PVI and an inverse correlation with AF burden. Apelin might be a promising marker of AF (Tab. 2, Fig. 2, Ref. 28).

Necessary but not sufficient: identifying conditions for effective feedback during internal medicine residents' clinical education

Competency-based medical education and programmatic assessment intend to increase the opportunities for meaningful feedback, yet these conversations remain elusive. By comparing resident and faculty perceptions of feedback opportunities within one internal medicine residency training program, we sought to understand whether and how principles underlying meaningful feedback could be supported or constrained across a variety of feedback opportunities. Using case-study qualitative methodology, interviews and focus groups were conducted to explore 19 internal medicine residents' and 7 faculty members' perceptions of feedback across a variety of feedback opportunities: coaching, mini-CEXs, in-training evaluation reports and routine clinical supervision. Our data analysis moved iteratively between developing conceptual understandings and fine-grained analyses, while attending to both deductive and inductive analysis. Our results suggest that all feedback opportunities, including those created through formalized assessments, can foster meaningful feedback if faculty establish a trusting relationship with the resident, base their feedback on direct observation and support resident learning. However, formalized assessments were often perceived as inhibiting the conditions for meaningful feedback. A coaching program provided a context in which meaningful feedback could arise, in part because faculty were supported in shifting their focus from patient to resident. Meaningful feedback in clinical education may be fostered across a variety of feedback opportunities, however, it is often constrained by assessment. We must consider whether increasing the frequency of formative assessments may inhibit efforts to improve our feedback cultures while, in contrast, freeing up faculty to focus on supporting resident learning could improve these cultures.

The nature of learning from simulation: Now I know it, now I'll do it, I'll work on that

CONTEXT

Ongoing learning in complex clinical environments requires health professionals to assess their own performance, manage their learning, and modify their practices based on self-monitored progress. Self-regulated learning (SRL) theory suggests that although learners may be capable of such learning, they often need guidance to enact it effectively. Debriefings following simulation may be an ideal time to support learners' use of SRL in targeted areas, but the extent to which they are optimally fostering these practices has not been examined.

METHODS

A qualitative study informed by grounded theory methodology was conducted in the context of three interprofessional in situ trauma simulations at our level 1 trauma centre. A total of 18 participants were interviewed both immediately and 5-6 weeks after the simulation experience. Transcripts were analysed using an iterative constant comparative approach to explore concepts and themes regarding the nature of learning from and after simulation.

RESULTS

During initial interviews, there were many examples of acquired content knowledge and straightforward practice changes that might not require ongoing SRL to enact well in practice. However, even for skills identified as needing to be 'worked on,' SRL strategies were lacking. At follow-up interviews, some participants had evolved more specific learning goals and rudimentary plans for implementation and improvement, but suggested this was prompted by the study interview questions rather than the simulation debriefing itself.

CONCLUSIONS

Overall, participants did not engage in fulsome development of SRL plans based on the simulation and debriefing; however, there were elements of SRL present, particularly after participants were given time to reflect on the interview questions and their own goals. This suggests that simulation training can support the use of SRL. However, debriefing approaches might be better optimised to take full advantage of the opportunity to encourage and foster SRL in practice after the simulation is over.

Learning After the Simulation Is Over: The Role of Simulation in Supporting Ongoing Self-Regulated Learning in Practice

The complex and dynamic nature of the clinical environment often requires health professionals to assess their own performance, manage their learning, and modify their practices based on self-monitored progress. Self-regulated learning studies suggest that while learners may be capable of such in situ learning, they often need guidance to enact it effectively. In this Perspective, the authors argue that simulation training may be an ideal venue to prepare learners for self-regulated learning in the clinical setting but may not currently be optimally fostering self-regulated learning practices. They point out that current simulation debriefing models emphasize the need to synthesize a set of identified goals for practice change (what behaviors might be modified) but do not address how learners might self-monitor the success of their implementation efforts and modify their learning plans based on this monitoring when back in the clinical setting. The authors describe the current models of simulation-based learning implied in the simulation literature and suggest potential targets in the simulation training process, which might be optimized to allow medical educators to take full advantage of the opportunity simulation provides to support and promote ongoing self-regulated learning in practice.

Learning With Patients, Students, and Peers: Continuing Professional Development in the Solo Practitioner Workplace

Conceptualizations of workplace learning have moved from knowledge acquisition to learning as participation in the practices and cultures of the workplace environment. Along with this has come an appreciation of applicability of sociocultural learning theories, which frame learning as occurring within "communities of practice" or learning being "situated" within a workplace environment where collaboration and social interaction are fundamental to the learning process. These conceptualizations of workplace learning are ideally suited to health professions where learners are supervised in clinical work environments and then continue to work in team-based environments as graduates. However, what happens to workplace learning for novice practitioners who have limited periods of clinical supervision and then graduate into solo or small group practices (which may also be in rural or remote locations) and embark on long working careers without supervision? This paper argues workplace learning needs to be scaffolded and supported to reach its full potential in these environments. Drawing on workplace-based learning theory, we highlight the ubiquitous nature of learning in the workplace, the importance of active engagement, reflection, and individual meaning making. Through this reframing of traditional notions of continuing professional development, we emphasize the importance of patients, students, and other practitioners as partners in workplace learning for solo practitioners. We also focus on the role of educators, professional associations, and regulators in helping solo practitioners recognize, access, and maximize the learning opportunities inherent in relatively isolated practice environments.

Integrated virtual and cadaveric dissection laboratories enhance first year medical students' anatomy experience: a pilot study

BACKGROUND

Radiology integration into medical anatomy courses is well established, but there is a paucity of literature on integrating virtual dissection into cadaveric dissection laboratories. Virtual dissection is the digital dissection of medical images on touchscreen anatomy visualization tables. The purpose of this pilot study was to investigate the feasibility of integrating virtual dissection into a first-year medical cadaver-based anatomy course and to assess students' overall attitude towards this new technology.

METHODS

All students in first-year medicine at a single medical school participated in this study (n = 292). Six virtual dissection laboratories, which focused on normal anatomy, were developed and integrated into a cadaver-based anatomy course. The virtual dissection table (VDT) was also integrated into the final anatomy spot exam. Following the course, students completed a short evidence-informed survey which was developed using a theoretical framework for curriculum evaluation. Numerical data were tabulated, and qualitative content analysis was performed on students' unstructured comments.

RESULTS

The survey response rate was 69.2% (n = 202/292). Most (78.7%) students reported that virtual dissection enhanced their understanding of the cadaveric anatomy and the clinical applications of anatomy. Most (73.8%) students also felt that the VDT was an effective use of the laboratory time. Thirteen narrative comments were collected, most of which (61.5%) identified strengths of the curriculum.

CONCLUSIONS

In this pilot study, students perceived that their learning was enhanced when virtual dissection was combined with a cadaver-based anatomy laboratory. This study demonstrates that there is potential for virtual dissection to augment cadaveric dissection in medical education.

Use of Bioresorbable Vascular Scaffold technology in treating coronary bifurcation lesions: A report about long-term clinical results and review of available literature

INTRODUCTION

BVS proved safe in humans. ABSORB trials showed them performing similar to Drug Eluting Stents in simple coronary interventions. We assessed a registry of 63 patients with bifurcation lesions, treated by BVS and followed their outcomes up-to 5 years.

METHODS

Patients who satisfied the inclusion criteria were included. Data about contact information, baseline characteristics, findings of coronary angiogram, details of their interventional treatment; short and long-term outcomes up till 5 years was collected.

RESULTS

Acute feasibility of implantation in bifurcation was high (98 %). Rate of stent thrombosis, acute or sub-acute, was 3.1 %. Rate of re-intervention was 38 %. The average time for an event to occur was 1.6±0.8 years. Over 5 years, 56 % had developed MACE. Patients with MACE were more likely females, hypertensive, smokers, with acute presentations (p=NS), and diabetic (72 % vs 33 % non-diabetic; p=0.002). Patients treated with hybrid strategy of BVS and DES were more likely to develop MACE (64 % vs 49 % for others; P=ns). Patients treated by simple provisional stenting were less likely to develop MACE (45 % vs 60.5 %; p=ns). The average SYNTAX score of MACE patients was 27 vs 20; p=0.06). Diabetes was independently associated with MACE. Hypertension was of borderline statistical significance (2-sided Log rank for Hypertension p=0.06, for Diabetes p=0.01).

DISCUSSION

The use of multiple stenting strategies to treat true bifurcation lesions using BVS is feasible with low rate of serious adverse events, albeit on the long run, the rate of re-intervention is high and stringent follow up is required (Tab. 7, Fig. 3, Ref. 37).

Entrustment Ratings in Internal Medicine Training: Capturing Meaningful Supervision Decisions or Just Another Rating?

The implementation of Entrustable Professional Activities has led to the simultaneous development of assessment based on a supervisor's entrustment of a learner to perform these activities without supervision. While entrustment may be intuitive when we consider the direct observation of a procedural task, the current implementation of rating scales for internal medicine's non-procedural tasks, based on entrustability, may not translate into meaningful learner assessment. In these Perspectives, we outline a number of potential concerns with ad hoc entrustability assessments in internal medicine post-graduate training: differences in the scope of procedural vs. non-procedural tasks, acknowledgement of the type of clinical oversight common within internal medicine, and the limitations of entrustment language. We point towards potential directions for inquiry that would require us to clarify the purpose of the entrustability assessment, reconsider each of the fundamental concepts of entrustment in internal medicine supervision and explore the use of descriptive rather than numeric assessment approaches.

How well is each learner learning? Validity investigation of a learning curve-based assessment approach for ECG interpretation

Learning curves can support a competency-based approach to assessment for learning. When interpreting repeated assessment data displayed as learning curves, a key assessment question is: "How well is each learner learning?" We outline the validity argument and investigation relevant to this question, for a computer-based repeated assessment of competence in electrocardiogram (ECG) interpretation. We developed an on-line ECG learning program based on 292 anonymized ECGs collected from an electronic patient database. After diagnosing each ECG, participants received feedback including the computer interpretation, cardiologist's annotation, and correct diagnosis. In 2015, participants from a single institution, across a range of ECG skill levels, diagnosed at least 60 ECGs. We planned, collected and evaluated validity evidence under each inference of Kane's validity framework. For Scoring, three cardiologists' kappa for agreement on correct diagnosis was 0.92. There was a range of ECG difficulty across and within each diagnostic category. For Generalization, appropriate sampling was reflected in the inclusion of a typical clinical base rate of 39% normal ECGs. Applying generalizability theory presented unique challenges. Under the Extrapolation inference, group learning curves demonstrated expert-novice differences, performance increased with practice and the incremental phase of the learning curve reflected ongoing, effortful learning. A minority of learners had atypical learning curves. We did not collect Implications evidence. Our results support a preliminary validity argument for a learning curve assessment approach for repeated ECG interpretation with deliberate and mixed practice. This approach holds promise for providing educators and researchers, in collaboration with their learners, with deeper insights into how well each learner is learning.

Beyond hands-on and hands-off: supervisory approaches and entrustment on the inpatient ward

CONTEXT

The concept of entrustment has garnered significant attention in medical specialties, despite variability in supervision styles and entrustment decisions. There is a need to further study the enactment of supervision on inpatient wards to inform competency-based assessment design.

METHODS

Attending physicians, while supervising on clinical teaching inpatient wards, were invited to describe a recent moment of enacting supervision with an internal medicine resident. Constructivist grounded theory guided data collection and analysis. Interview transcripts were analysed in iterative cycles to inform data collection. Constant comparison was used to build a theory of supervision from the identified themes.

RESULTS

In 2016-2017, 23 supervisors from two Canadian universities with supervision reputations ranging from very involved to less involved participated in one or two interviews (total: 28). Supervisors were not easily dichotomised into styles based on behaviour because all used similar oversight strategies. Supervisors described adjusting between 'hands-on' (e.g. detail oriented) and 'hands-off' (e.g. less visible on ward) styles depending on the context. All also contended with the competing roles of clinical teacher and care provider. Supervisors made a distinction between the terms `entrust' and `trust', and did not grant complete entrustment to senior residents.

CONCLUSIONS

We propose that a supervisor's perceived responsibility for the ward underlies adjustments between 'hands-on' (i.e. personal ward responsibility) and 'hands-off' (i.e. shared ward responsibility) styles. Our approaches to clinical supervision model combines this responsibility tension with the tension between patient care and teaching to illustrate four supervisory approaches, each with unique priorities influencing entrustment. Given the fluidity in supervision, documenting changes in oversight strategies, rather than absolute levels of entrustment, may be more informative for assessment purposes. Research is needed to determine if there is sufficient association between the supervision provided, the entrustment decision made and the supervisor's trust in a trainee to use these as proxies in assessing a trainee's competence.

Structural assessment of myocardial infarction scars and left ventricular function with cardiac magnetic resonance imaging in patients at high risk of sudden cardiac death

OBJECTIVES

Results of qualitative and quantitative analyses of scars and LV (left ventricle) function acquired by means of cardiac magnetic resonance (CMR) were correlated with a subsequent occurrence of malignant ventricular arrhythmias in patients at high risk of sudden cardiac death (SCD).

METHODS

We have prospectively followed 47 patients (mean age 60 ± 11 years) who were hospitalized for an implantable cardioverter-defibrillator (ICD) implantation to prevent SCD. All post-MI patients had severe residual LV dysfunction (LVEF 33±14%). Patients were examined with CMR. Based on CMR analysis, we evaluated the basic functional parameters of LV as well as mass, volume, transmurality and heterogeneity of the post-MI scar.

RESULTS

The patients with malignant arrhythmias were characterized by smaller LV end-diastolic diameters (LVED 192 ± 79 vs 254 ± 47 mm, p = 0.003) and end-systolic diameters (LVES 131 ± 80 vs 181 ± 45 mm, p = 0.01). As for the other observed functional and morphological CMR parameters, no significant differences between the two groups were detected.

CONCLUSION

These results indicate that post-MI patients with severe residual left ventricular dysfunction and dilatation are in the long term characterized by a lower incidence of malignant arrhythmias compared to the patients with less dilated LV with a comparably severe LV dysfunction (Tab. 2, Fig. 3, Ref. 26). Text in PDF www.elis.sk.

Core Competencies or a Competent Core? A Scoping Review and Realist Synthesis of Invasive Bedside Procedural Skills Training in Internal Medicine

PURPOSE

Invasive bedside procedures are core competencies for internal medicine, yet no formal training guidelines exist. The authors conducted a scoping review and realist synthesis to characterize current training for lumbar puncture, arthrocentesis, paracentesis, thoracentesis, and central venous catheterization. They aimed to collate how educators justify using specific interventions, establish which interventions have the best evidence, and offer directions for future research and training.

METHOD

The authors systematically searched Medline, Embase, the Cochrane Library, and ERIC through April 2015. Studies were screened in three phases; all reviews were performed independently and in duplicate. The authors extracted information on learner and patient demographics, study design and methodological quality, and details of training interventions and measured outcomes. A three-step realist synthesis was performed to synthesize findings on each study's context, mechanism, and outcome, and to identify a foundational training model.

RESULTS

From an initial 6,671 studies, 149 studies were further reduced to 67 (45%) reporting sufficient information for realist synthesis. Analysis yielded four types of procedural skills training interventions. There was relative consistency across contexts and significant differences in mechanisms and outcomes across the four intervention types. The medical procedural service was identified as an adaptable foundational training model.

CONCLUSIONS

The observed heterogeneity in procedural skills training implies that programs are not consistently developing residents who are competent in core procedures. The findings suggest that researchers in education and quality improvement will need to collaborate to design training that develops a "competent core" of proceduralists using simulation and clinical rotations.

Guidelines: The do's, don'ts and don't knows of direct observation of clinical skills in medical education

INTRODUCTION

Direct observation of clinical skills is a key assessment strategy in competency-based medical education. The guidelines presented in this paper synthesize the literature on direct observation of clinical skills. The goal is to provide a practical list of Do's, Don'ts and Don't Knows about direct observation for supervisors who teach learners in the clinical setting and for educational leaders who are responsible for clinical training programs.

METHODS

We built consensus through an iterative approach in which each author, based on their medical education and research knowledge and expertise, independently developed a list of Do's, Don'ts, and Don't Knows about direct observation of clinical skills. Lists were compiled, discussed and revised. We then sought and compiled evidence to support each guideline and determine the strength of each guideline.

RESULTS

A final set of 33 Do's, Don'ts and Don't Knows is presented along with a summary of evidence for each guideline. Guidelines focus on two groups: individual supervisors and the educational leaders responsible for clinical training programs. Guidelines address recommendations for how to focus direct observation, select an assessment tool, promote high quality assessments, conduct rater training, and create a learning culture conducive to direct observation.

CONCLUSIONS

High frequency, high quality direct observation of clinical skills can be challenging. These guidelines offer important evidence-based Do's and Don'ts that can help improve the frequency and quality of direct observation. Improving direct observation requires focus not just on individual supervisors and their learners, but also on the organizations and cultures in which they work and train. Additional research to address the Don't Knows can help educators realize the full potential of direct observation in competency-based education.

Using In-Training Evaluation Report (ITER) Qualitative Comments to Assess Medical Students and Residents: A Systematic Review

PURPOSE

In-training evaluation reports (ITERs) constitute an integral component of medical student and postgraduate physician trainee (resident) assessment. ITER narrative comments have received less attention than the numeric scores. The authors sought both to determine what validity evidence informs the use of narrative comments from ITERs for assessing medical students and residents and to identify evidence gaps.

METHOD

Reviewers searched for relevant English-language studies in MEDLINE, EMBASE, Scopus, and ERIC (last search June 5, 2015), and in reference lists and author files. They included all original studies that evaluated ITERs for qualitative assessment of medical students and residents. Working in duplicate, they selected articles for inclusion, evaluated quality, and abstracted information on validity evidence using Kane's framework (inferences of scoring, generalization, extrapolation, and implications).

RESULTS

Of 777 potential articles, 22 met inclusion criteria. The scoring inference is supported by studies showing that rich narratives are possible, that changing the prompt can stimulate more robust narratives, and that comments vary by context. Generalization is supported by studies showing that narratives reach thematic saturation and that analysts make consistent judgments. Extrapolation is supported by favorable relationships between ITER narratives and numeric scores from ITERs and non-ITER performance measures, and by studies confirming that narratives reflect constructs deemed important in clinical work. Evidence supporting implications is scant.

CONCLUSIONS

The use of ITER narratives for trainee assessment is generally supported, except that evidence is lacking for implications and decisions. Future research should seek to confirm implicit assumptions and evaluate the impact of decisions.

Screening for Atrial Fibrillation

Approximately 10% of ischemic strokes are associated with atrial fibrillation (AF) first diagnosed at the time of stroke. Detecting asymptomatic AF would provide an opportunity to prevent these strokes by instituting appropriate anticoagulation. The AF-SCREEN international collaboration was formed in September 2015 to promote discussion and research about AF screening as a strategy to reduce stroke and death and to provide advocacy for implementation of country-specific AF screening programs. During 2016, 60 expert members of AF-SCREEN, including physicians, nurses, allied health professionals, health economists, and patient advocates, were invited to prepare sections of a draft document. In August 2016, 51 members met in Rome to discuss the draft document and consider the key points arising from it using a Delphi process. These key points emphasize that screen-detected AF found at a single timepoint or by intermittent ECG recordings over 2 weeks is not a benign condition and, with additional stroke factors, carries sufficient risk of stroke to justify consideration of anticoagulation. With regard to the methods of mass screening, handheld ECG devices have the advantage of providing a verifiable ECG trace that guidelines require for AF diagnosis and would therefore be preferred as screening tools. Certain patient groups, such as those with recent embolic stroke of uncertain source (ESUS), require more intensive monitoring for AF. Settings for screening include various venues in both the community and the clinic, but they must be linked to a pathway for appropriate diagnosis and management for screening to be effective. It is recognized that health resources vary widely between countries and health systems, so the setting for AF screening should be both country- and health system-specific. Based on current knowledge, this white paper provides a strong case for AF screening now while recognizing that large randomized outcomes studies would be helpful to strengthen the evidence base.

Staging a performance: learners' perceptions about direct observation during residency

CONTEXT

Evidence strongly supports that direct observation is a valid and reliable assessment tool; support for its impact on learning is less compelling, and we know that some learners are ambivalent about being observed. However, learners' perceptions about the impact of direct observation on their learning and professional development remain underexplored. To promote learning, we need to understand what makes direct observation valuable for learners.

METHODS

Informed by constructivist grounded theory, we interviewed 22 learners about their observation experiences. Data collection and analysis occurred iteratively; themes were identified using constant comparative analysis.

RESULTS

Direct observation was widely endorsed as an important educational strategy, albeit one that created significant anxiety. Opaque expectations exacerbated participants' discomfort, and participants described that being observed felt like being assessed. Consequently, participants exchanged their 'usual' practice for a 'textbook' approach; alterations to performance generated uncertainty about their role, and raised questions about whether observers saw an authentic portrayal of their knowledge and skill.

CONCLUSION

An 'observer effect' may partly explain learners' ambivalence about direct observation; being observed seemed to magnify learners' role ambiguity, intensify their tensions around professional development and raise questions about the credibility of feedback. In turn, an observer effect may impact learners' receptivity to feedback and may explain, in part, learners' perceptions that useful feedback is scant. For direct observation to be valuable, educators must be explicit about expectations, and they must be aware that how learners perform in the presence of an observer may not reflect what they do as independent practitioners. To nurture learners' professional development, educators must create a culture of observation-based coaching that is divorced from assessment and is tailored to developing learners' identities as practitioners of both the art and the science of medicine.

Validation of educational assessments: a primer for simulation and beyond

BACKGROUND

Simulation plays a vital role in health professions assessment. This review provides a primer on assessment validation for educators and education researchers. We focus on simulation-based assessment of health professionals, but the principles apply broadly to other assessment approaches and topics.

KEY PRINCIPLES

Validation refers to the process of collecting validity evidence to evaluate the appropriateness of the interpretations, uses, and decisions based on assessment results. Contemporary frameworks view validity as a hypothesis, and validity evidence is collected to support or refute the validity hypothesis (i.e., that the proposed interpretations and decisions are defensible). In validation, the educator or researcher defines the proposed interpretations and decisions, identifies and prioritizes the most questionable assumptions in making these interpretations and decisions (the "interpretation-use argument"), empirically tests those assumptions using existing or newly-collected evidence, and then summarizes the evidence as a coherent "validity argument." A framework proposed by Messick identifies potential evidence sources: content, response process, internal structure, relationships with other variables, and consequences. Another framework proposed by Kane identifies key inferences in generating useful interpretations: scoring, generalization, extrapolation, and implications/decision. We propose an eight-step approach to validation that applies to either framework: Define the construct and proposed interpretation, make explicit the intended decision(s), define the interpretation-use argument and prioritize needed validity evidence, identify candidate instruments and/or create/adapt a new instrument, appraise existing evidence and collect new evidence as needed, keep track of practical issues, formulate the validity argument, and make a judgment: does the evidence support the intended use?

CONCLUSIONS

Rigorous validation first prioritizes and then empirically evaluates key assumptions in the interpretation and use of assessment scores. Validation science would be improved by more explicit articulation and prioritization of the interpretation-use argument, greater use of formal validation frameworks, and more evidence informing the consequences and implications of assessment.

'Sometimes the work just needs to be done': socio-cultural influences on direct observation in medical training

CONTEXT

Direct observation promises to strengthen both coaching and assessment, and calls for its increased use in medical training abound. Despite its apparent potential, the uptake of direct observation in medical training remains surprisingly limited outside the formal assessment setting. The limited uptake of observation raises questions about cultural barriers to its use. In this study, we explore the influence of professional culture on the use of direct observation within medical training.

METHODS

Using a constructivist grounded theory approach, we interviewed 22 residents or fellows (10 male, 12 female) about their experiences of being observed during training. Participants represented a range of specialties and training levels. Data collection and analysis were conducted iteratively. Themes were identified using constant comparative analysis.

RESULTS

Observation was used selectively; specialties tended to observe the clinical acts that they valued most. Despite these differences, we found two cultural values that consistently challenged the ready implementation of direct observation across specialties: (i) autonomy in learning and (ii) efficiency in health care provision. Furthermore, we found that direct observation was a primarily learner-driven activity, which left learners caught in the middle, wanting observation but also wanting to appear independent and efficient.

CONCLUSIONS

The cultural values of autonomy in learning and practice and efficiency in health care provision challenge the integration of direct observation into clinical training. Medical learners are often expected to ask for observation, but such requests are socially and culturally fraught, and likely to constrain the wider uptake of direct observation.

When Assessment Data Are Words: Validity Evidence for Qualitative Educational Assessments

Quantitative scores fail to capture all important features of learner performance. This awareness has led to increased use of qualitative data when assessing health professionals. Yet the use of qualitative assessments is hampered by incomplete understanding of their role in forming judgments, and lack of consensus in how to appraise the rigor of judgments therein derived. The authors articulate the role of qualitative assessment as part of a comprehensive program of assessment, and translate the concept of validity to apply to judgments arising from qualitative assessments. They first identify standards for rigor in qualitative research, and then use two contemporary assessment validity frameworks to reorganize these standards for application to qualitative assessment.Standards for rigor in qualitative research include responsiveness, reflexivity, purposive sampling, thick description, triangulation, transparency, and transferability. These standards can be reframed using Messick's five sources of validity evidence (content, response process, internal structure, relationships with other variables, and consequences) and Kane's four inferences in validation (scoring, generalization, extrapolation, and implications). Evidence can be collected and evaluated for each evidence source or inference. The authors illustrate this approach using published research on learning portfolios.The authors advocate a "methods-neutral" approach to assessment, in which a clearly stated purpose determines the nature of and approach to data collection and analysis. Increased use of qualitative assessments will necessitate more rigorous judgments of the defensibility (validity) of inferences and decisions. Evidence should be strategically sought to inform a coherent validity argument.

Investigating conditions for meaningful feedback in the context of an evidence-based feedback programme

CONTEXT

We developed, implemented and evaluated an evidence-based programme of feedback designed to address limitations identified in the current literature.

OBJECTIVES

We sought to advance understanding about how and why feedback processes might be more effective in clinical education.

METHODS

Three faculty members and nine first-year internal medicine residents participated in the pilot programme. To counter challenges identified in the literature, feedback was based on direct observation, grounded in longitudinal faculty-resident relationships, and devoid of summative assessment. We used a qualitative case study design to address three research questions: (i) What benefits did the participants describe? (ii) What elements of the programme facilitated these benefits? (iii) What were the limitations and challenges of the programme? Collected data included audiotapes of interactions between faculty members and residents, field notes written during observations, and semi-structured interviews and focus groups with resident participants. Data analysis moved cyclically and iteratively through inductive and deductive analysis.

RESULTS

Residents described benefits relating to their ways of working (clinical skills), ways of learning (accountability for learning) and ways of feeling (emotional well-being). According to participants, specific elements of the programme that achieved these benefits included the direct observation of authentic clinical work, the longitudinal relationship with a faculty member and the emergence of feedback as a conversation between the faculty member and learner.

CONCLUSIONS

We conclude that the conditions established within our pilot feedback programme influenced the learning culture for first-year internal medicine residents by grounding direct observation in authentic clinical work and setting the observations in the context of a longitudinal, non-assessment-based relationship between a faculty member and resident. These conditions appeared to influence residents' participation in the feedback process, their ways of approaching their daily clinical work, their emotional well-being and their engagement in their own learning.

Examining Residents' Strategic Mindfulness During Self-Regulated Learning of a Simulated Procedural Skill

BACKGROUND

Simulation-based training is currently embedded in most health professions education curricula. Without evidence for how trainees think about their simulation-based learning, some training techniques may not support trainees' learning strategies.

OBJECTIVE

This study explored how residents think about and self-regulate learning during a lumbar puncture (LP) training session using a simulator.

METHODS

In 2010, 20 of 45 postgraduate year 1 internal medicine residents attended a mandatory procedural skills training boot camp. Independently, residents practiced the entire LP skill on a part-task trainer using a clinical LP tray and proper sterile technique. We interviewed participants regarding how they thought about and monitored their learning processes, and then we conducted a thematic analysis of the interview data.

RESULTS

The analysis suggested that participants considered what they could and could not learn from the simulator; they developed their self-confidence by familiarizing themselves with the LP equipment and repeating the LP algorithmic steps. Participants articulated an idiosyncratic model of learning they used to interpret the challenges and successes they experienced. Participants reported focusing on obtaining cerebrospinal fluid and memorizing the "routine" version of the LP procedure. They did not report much thinking about their learning strategies (eg, self-questioning).

CONCLUSIONS

During simulation-based training, residents described assigning greater weight to achieving procedural outcomes and tended to think that the simulated task provided them with routine, generalizable skills. Over this typical 1-hour session, trainees did not appear to consider their strategic mindfulness (ie, awareness and use of learning strategies).