Students' educational needs for clinical reasoning in first clerkships

Identifying the response process validity of clinical vignette-type multiple choice questions: An eye-tracking study

INTRODUCTION

Clinical vignette-type multiple choice questions (CV-MCQs) are widely used in assessment and identifying the response process validity (RPV) of questions with low and high integration of knowledge is essential. Answering CV-MCQs of different levels of knowledge application and integration can be understood from a cognitive workload perspective and this can be identified by using eye-tracking. The aim of the pilot study was to identify the cognitive workload and RPV of CV-MCQs of different levels of knowledge application and integration by the use eye-tracking.

METHODS

Fourteen fourth-year medical students answered a test with 40 CV-MCQs, which were equally divided into low-level and high-level complexity (knowledge application and integration). Cognitive workload was measured using screen-based eye tracking, with the number of fixations and revisitations for each area of interest.

RESULTS

We found a higher cognitive workload for high-level complexity (M = 121.74) compared with lower-level complexity questions (M = 51.94) and also for participants who answered questions incorrectly (M = 94.31) compared with correctly (M = 79.36).

CONCLUSION

Eye-tracking has the potential to become a useful and practical approach for helping to identify the RPV of CV-MCQs. This approach can be used for improving the design and development of CV-MCQs, and to provide feedback to inform teaching and learning.[Box: see text].

Enhancing our ability to diagnose cardiac valve disease by applying a graphical educational game

Objectives

This study aimed to compare a newly designed graphical educational game (GEG) with a case-based learning (CBL) exercise and to enhance our ability to apply physiological knowledge of the cardiac cycle to diagnose cardiac valvular diseases among preclinical medical students.

Methods

In this interventional study, first-year undergraduate medical students were randomly assigned to a GEG group (n = 42) and a CBL group (n = 37). The GEG group involved shading cardiac cycle graphs and pressure–volume loops while the CBL group worked on two cases of cardiac valve diseases. A multiple-choice question (MCQ) test was then used to assess conceptual understanding of the cardiac cycle. After brief exposure to murmur auscultation on a simulator manikin, the groups were assessed in a simulator manikin test for their ability to diagnose cardiac valve disease. Median MCQ scores and mean scores in the simulator test were then compared using the Mann–Whitney U test. The student's perspectives of the GEG and simulation session were acquired on a 5-point Likert scale questionnaire.

Results

The GEG group had significantly higher median MCQ scores (p < 0.001) and mean simulator test scores (p < 0.001) when compared to the CBL group. Moreover, 91% of students agreed that the GEG helped them to clarify concepts, and 88% agreed that the concepts and knowledge gained through the GEG helped them to diagnose valve disease in the manikins.

Conclusion

The GEG was positively received by students and was more useful than the CBL in enhancing the application of cardiac physiology concepts and improving diagnostic ability in a simulated clinical setting.

Barriers to Learning Clinical Reasoning: a Qualitative Study of Medicine Clerkship Students

Clinical reasoning is crucial to good patient care, but both learning and applying clinical reasoning skills in the context of a complex working environment can be challenging. We sought to understand the perceived barriers to learning clinical reasoning, as experienced by internal medicine clerkship students at our institution. We invited internal medicine clerkship students to participate in focus groups to discuss their experiences with and barriers to learning clinical reasoning. A survey was administered to gather additional responses. Responses were reviewed, coded, and synthesized to identify key themes. Twenty-nine medicine clerkship students (male = 14, female = 15) participated in six 60-minute focus groups, and 121 (61% response rate) students responded to the barriers to clinical reasoning survey from March 2018 to May 2019. We identified three themes (clerkship acclimation, data access, and practice optimization) and ten subthemes as aspects of the clerkship environment that impacted students' ability to develop clinical reasoning skills. Students identified barriers to learning clinical reasoning during the internal medicine clerkship. The themes "clerkship acclimation" and "data access" were identified as prerequisites to clinical reasoning while the theme "practice optimization" described key components of the deliberate practice of clinical reasoning. Educators and health systems may improve the development of clinical reasoning by recognizing and overcoming these barriers within clinical learning environments.

Clinical reasoning as a threshold skill

Background: Threshold skills are defined as new ways of thinking about and performing in a discipline. They represent transformed ways of thinking and doing that are pivotal to learners' progress. Our aim was to establish whether clinical reasoning exhibited features of a threshold skill. Methods: Twenty-five final-year medical students were interviewed with a five-question protocol about how they were learning clinical reasoning. Students' responses were analyzed using a deductive method to identify features of threshold skills. Results: Students' descriptions of learning clinical reasoning exhibited five features: transformation, troublesomeness, integration, association with practice, and issues with transferability. Conclusions: Viewing clinical reasoning as a threshold skill is a novel interpretation of its nature and has implications for learning, teaching, and research. Students can be reassured that, although initially troublesome, with practice, they will not only learn the skill but also how to use it more effectively. Teachers can help students to understand that clinical reasoning is difficult to learn and will require time and repeated practice under supervision to develop.

Voluntary attendance of small-group brainstorming tutoring courses intensify new clerk's "excellence in clinical care": a pilot study

BACKGROUND

Clerkship provides a unique way of transferring the knowledge and skills gathered during medical school's curriculum into real-ward clinical care environment. The annual program evaluation has indicated that the training of clerks in diagnostic and clinical reasoning skills needed to be enhanced. Recently, "clinical excellence" program have been promoted in our institution to augment the excellence in clinical care of new clerks. Current study aims to evaluate whether this pilot program improve the "clinical excellence" of new clerks.

METHODS

In a pilot study, groups of new clerks in years 2013 and 2014 voluntarily attended either a small-group brainstorming course or a didactic classroom tutoring courses as part of their 3-month internal medicine clinical rotation block. A third group of new clerks did not join either of the above courses and this group served as the control group. Pre-block/post-block self-assessment and post-block 5-station mini-Objective Subjective Clinical Examinations (OSCEs) were used to evaluate the effectiveness of these two additional courses that trained diagnostic and clinical reasoning skills.

RESULTS

Overtime, the percentages of new clerks that attended voluntarily either the small-group brainstorming or classroom tutoring courses were increased. Higher post-block self-assessed diagnostic and clinical reasoning skill scores were found among individuals who attended the small-group brainstorming courses compared to either the didactic group or the control group. In a corresponding manner, the small-group brainstorming group obtained higher summary OSCE_diag and OSCE_reason scores than either the didactic group or control group. For all basic images/laboratory OSCE stations, the individual diagnostic skill (OSCE_diag) scores of the small-group brainstorming group were higher than those of the didactic group. By way of contrast, only the clinical reasoning skill (OSCE_reason) scores of the basic electrocardiogram and complete blood count + biochemistry OSCE station of thesmall-group brainstorming group were higher than those of the didactic group. Among the small-group brainstorming group, clerks with higher cumulative learning hours (>30-h) had significant higher OSCE_diag and OSCE_reason scores (>400) than those with less cumulative learning hours.

CONCLUSION

Our pilot study provides a successful example of the use of a small-group tutoring courses for augmenting the diagnostic and clinical reasoning skills of new clerks. The positive results obtained during the initial 2-year long pilot "clinical excellence" program have encouraged the formal implementation of this course as part of the clerkship curriculum.

Validation of undergraduate medical student script concordance test (SCT) scores on the clinical assessment of the acute abdomen

Background

Health professionals often manage medical problems in critical situations under time pressure and on the basis of vague information. In recent years, dual process theory has provided a framework of cognitive processes to assist students in developing clinical reasoning skills critical especially in surgery due to the high workload and the elevated stress levels. However, clinical reasoning skills can be observed only indirectly and the corresponding constructs are difficult to measure in order to assess student performance. The script concordance test has been established in this field. A number of studies suggest that the test delivers a valid assessment of clinical reasoning. However, different scoring methods have been suggested. They reflect different interpretations of the underlying construct. In this work we want to shed light on the theoretical framework of script theory and give an idea of script concordance testing. We constructed a script concordance test in the clinical context of “acute abdomen” and compared previously proposed scores with regard to their validity.

Methods

A test comprising 52 items in 18 clinical scenarios was developed, revised along the guidelines and administered to 56 4^th and 5^th year medical students at the end of a blended-learning seminar. We scored the answers using five different scoring methods (distance (2×), aggregate (2×), single best answer) and compared the scoring keys, the resulting final scores and Cronbach’s α after normalization of the raw scores.

Results

All scores except the single best answers calculation achieved acceptable reliability scores (>= 0.75), as measured by Cronbach’s α. Students were clearly distinguishable from the experts, whose results were set to a mean of 80 and SD of 5 by the normalization process. With the two aggregate scoring methods, the students’ means values were between 62.5 (AGGPEN) and 63.9 (AGG) equivalent to about three expert SD below the experts’ mean value (Cronbach’s α : 0.76 (AGGPEN) and 0.75 (AGG)). With the two distance scoring methods the students’ mean was between 62.8 (DMODE) and 66.8 (DMEAN) equivalent to about two expert SD below the experts’ mean value (Cronbach’s α: 0.77 (DMODE) and 0.79 (DMEAN)). In this study the single best answer (SBA) scoring key yielded the worst psychometric results (Cronbach’s α: 0.68).

Conclusion

Assuming the psychometric properties of the script concordance test scores are valid, then clinical reasoning skills can be measured reliably with different scoring keys in the SCT presented here. Psychometrically, the distance methods seem to be superior, wherein inherent statistical properties of the scales might play a significant role. For methodological reasons, the aggregate methods can also be used. Despite the limitations and complexity of the underlying scoring process and the calculation of reliability, we advocate for SCT because it allows a new perspective on the measurement and teaching of cognitive skills.

Students' performance in the different clinical skills assessed in OSCE: what does it reveal?

INTRODUCTION

The purpose of this study was to compare students' performance in the different clinical skills (CSs) assessed in the objective structured clinical examination.

METHODS

Data for this study were obtained from final year medical students' exit examination (n=185). Retrospective analysis of data was conducted using SPSS. Means for the six CSs assessed across the 16 stations were computed and compared.

RESULTS

Means for history taking, physical examination, communication skills, clinical reasoning skills (CRSs), procedural skills (PSs), and professionalism were 6.25±1.29, 6.39±1.36, 6.34±0.98, 5.86±0.99, 6.59±1.08, and 6.28±1.02, respectively. Repeated measures ANOVA showed there was a significant difference in the means of the six CSs assessed [F(2.980, 548.332)=20.253, p<0.001]. Pairwise multiple comparisons revealed significant differences between the means of the eight pairs of CSs assessed, at p<0.05.

CONCLUSIONS

CRSs appeared to be the weakest while PSs were the strongest, among the six CSs assessed. Students' unsatisfactory performance in CRS needs to be addressed as CRS is one of the core competencies in medical education and a critical skill to be acquired by medical students before entering the workplace. Despite its challenges, students must learn the skills of clinical reasoning, while clinical teachers should facilitate the clinical reasoning process and guide students' clinical reasoning development.

PME aiming for quality, originality and peer recognition

None