Types of clinical reasoning in a summative clerkship oral examination

INTRODUCTION

Dual-process theory characterizes clinical reasoning (CR) as Type 1 (intuitive) and Type 2 (analytical) thinking. This study examined CR on a summative clinical clerkship structured clinical oral examination (SCOE).

METHODS AND SUBJECTS

511 clinical clerks at the University of Toronto underwent SCOEs. Type 1, Type 2, and Global CR performance were compared to other internal medicine clerkship assessments using descriptive statistics and Spearman correlations.

RESULTS

Clinical clerks achieved mean marks >75% on the three clinical reasoning stations, on Type 1 and 2 CR tasks, and the overall SCOE. Performance on the SCOE CR stations correlated with each of the other clerkship assessments: written examination, inpatient, and ambulatory clinic assessments. The correlation of performance between Type 1 and Type 2 clinical reasoning tasks was statistically significant but weak (r_s = 0.28). This suggests that defined measures of Type 1 and Type 2 reasoning were indeed assessing distinct constructs.

CONCLUSION

Clinical clerks used both Type 1 and Type 2 reasoning with success. This study's characterization of Type 1 and Type 2 CR as separate domains, distinct from existing measures on the SCOE as well as the other clerkship assessments, can suggest a further addition to multimodal clerkship assessment.

Gathering Validity Evidence on an Internal Medicine Clerkship Multistep Exam to Assess Medical Student Analytic Ability

Construct: The definition of clinical reasoning may vary among health profession educators. However, for the purpose of this paper, clinical reasoning is defined as the cognitive processes that are involved in the steps of information gathering, problem representation, generating a differential diagnosis, providing a diagnostic justification to arrive at a leading diagnosis, and formulating diagnostic and management plans. Background: Expert performance in clinical reasoning is essential for success as a physician, and has been difficult for clerkship directors to observe and quantify in a way that fosters the instruction and assessment of clinical reasoning. The purpose of this study was to gather validity evidence for the Multistep exam (MSX) format used by our medicine clerkship to assess analytical clinical reasoning abilities; we did this by examining the relationship between scores on the MSX and other external measures of clinical reasoning abilities. This analysis used dual process theory as the main theoretical framework of clinical reasoning, as well as aspects of Kane's validity framework to guide the selection of validity evidence for the investigation. We hypothesized that there would be an association between the MSX (a three-step clinical reasoning tool developed locally), and the USMLE Step 2 CS, as they share similar concepts in assessing the clinical reasoning of students. We examined the relationship between overall scores on the MSX and the Step 2 CS Integrated Clinical Encounter (ICE) score, in which the student articulates their reasoning for simulated patient cases, while controlling for examinee's internal medicine clerkship performance measures such as the NBME subject exam score and the Medicine clerkship OSCE score. Approach: A total 477 of 487 (97.9%) medical students, representing the graduating classes of 2015, 2016, 2017, who took the MSX at the end of each medicine clerkship (2012-2016), and Step 2 CS (2013-2017) were included in this study. Correlation analysis and multiple linear regression analysis were used to examine the impact of the primary explanatory variables of interest (MSX) onto the outcome variable (ICE score) when controlling for baseline variables (Medicine OSCE and NBME Medicine subject exam). Findings: The overall MSX score had a significant, positive correlation with the Step 2 CS ICE score (r = .26, P < .01). The overall MSX score was a significant predictor of Step 2 CS ICE score (ß = .19, P < .001), explaining an additional 4% of the variance of ICE beyond the NBME Medicine subject score and the Medicine OSCE score (Adjusted R² = 13%). Conclusion: The stepwise format of the MSX provides a tool to observe clinical reasoning performance, which can be used in an assessment system to provide feedback to students on their analytical clinical reasoning. Future studies should focus on gaining additional validity evidence across different learners and multiple medical schools.

Regular Formal Evaluation Sessions are Effective as Frame-of-Reference Training for Faculty Evaluators of Clerkship Medical Students

BACKGROUND

Face-to-face formal evaluation sessions between clerkship directors and faculty can facilitate the collection of trainee performance data and provide frame-of-reference training for faculty.

OBJECTIVE

We hypothesized that ambulatory faculty who attended evaluation sessions at least once in an academic year (attendees) would use the Reporter-Interpreter-Manager/Educator (RIME) terminology more appropriately than faculty who did not attend evaluation sessions (non-attendees).

DESIGN

Investigators conducted a retrospective cohort study using the narrative assessments of ambulatory internal medicine clerkship students during the 2008-2009 academic year.

PARTICIPANTS

The study included assessments of 49 clerkship medical students, which comprised 293 individual teacher narratives.

MAIN MEASURES

Single-teacher written and transcribed verbal comments about student performance were masked and reviewed by a panel of experts who, by consensus, (1) determined whether RIME was used, (2) counted the number of RIME utterances, and (3) assigned a grade based on the comments. Analysis included descriptive statistics and Pearson correlation coefficients.

KEY RESULTS

The authors reviewed 293 individual teacher narratives regarding the performance of 49 students. Attendees explicitly used RIME more frequently than non-attendees (69.8 vs. 40.4 %; p < 0.0001). Grades recommended by attendees correlated more strongly with grades assigned by experts than grades recommended by non-attendees (r = 0.72; 95 % CI (0.65, 0.78) vs. 0.47; 95 % CI (0.26, 0.64); p = 0.005). Grade recommendations from individual attendees and non-attendees each correlated significantly with overall student clerkship clinical performance [r = 0.63; 95 % CI (0.54, 0.71) vs. 0.52 (0.36, 0.66), respectively], although the difference between the groups was not statistically significant (p = 0.21).

CONCLUSIONS

On an ambulatory clerkship, teachers who attended evaluation sessions used RIME terminology more frequently and provided more accurate grade recommendations than teachers who did not attend. Formal evaluation sessions may provide frame-of-reference training for the RIME framework, a method that improves the validity and reliability of workplace assessment.

The Long-Term Career Outcome Study: Lessons Learned and Implications for Educational Practice

The work of the Long-Term Career Outcome Study has been a program of scholarship spanning 10 years. Borrowing from established quality assurance literature, the Long-Term Career Outcome Study team has organized its scholarship into three phases; before medical school, during medical school, and after medical school. The purpose of this commentary is to address two fundamental questions: (1) what has been learned? and (2) how does this knowledge translate to educational practice and policy now and into the future? We believe that answers to these questions are relevant not only to our institution but also to other educational institutions seeking to provide high-quality health professions education.