It's the destination: diagnostic accuracy and reasoning

Reliability, validity and acceptability of an online clinical reasoning simulator for medical students: An international pilot

INTRODUCTION

Clinical reasoning skills are essential for decision-making. Current assessment methods are limited when testing clinical reasoning and management of uncertainty. This study evaluates the reliability, validity and acceptability of Practicum Script, an online simulation-based programme, for developing medical students' clinical reasoning skills using real-life cases.

METHODS

In 2020, we conducted an international, multicentre pilot study using 20 clinical cases with 2457 final-year medical students from 21 schools worldwide. Psychometric analysis was performed (n = 1502 students completing at least 80% of cases). Classical estimates of reliability for three test domains (hypothesis generation, hypothesis argumentation and knowledge application) were calculated using Cronbach's alpha and McDonald's omega coefficients. Validity evidence was obtained by confirmatory factor analysis (CFA) and measurement alignment (MA). Items from the knowledge application domain were analysed using cognitive diagnostic modelling (CDM). Acceptability was evaluated by an anonymous student survey.

RESULTS

Reliability estimates were high with narrow confidence intervals. CFA revealed acceptable goodness-of-fit indices for the proposed three-factor model. CDM analysis demonstrated good absolute test fit and high classification accuracy estimates. Student survey responses showed high levels of acceptability.

CONCLUSION

Our findings suggest that Practicum Script is a useful resource for strengthening students' clinical reasoning skills and ability to manage uncertainty.

Learning strategy impacts medical diagnostic reasoning in early learners

Relating learned information to similar yet new scenarios, transfer of learning, is a key characteristic of expert reasoning in many fields including medicine. Psychological research indicates that transfer of learning is enhanced via active retrieval strategies. For diagnostic reasoning, this finding suggests that actively retrieving diagnostic information about patient cases could improve the ability to engage in transfer of learning to later diagnostic decisions. To test this hypothesis, we conducted an experiment in which two groups of undergraduate student participants learned symptom lists of simplified psychiatric diagnoses (e.g., Schizophrenia; Mania). Next, one group received written patient cases and actively retrieved the cases from memory and the other group read these written cases twice, engaging in a passive rehearsal learning strategy. Both groups then diagnosed test cases that had two equally valid diagnoses-one supported by "familiar" symptoms described in learned patient cases, and one by novel symptom descriptions. While all participants were more likely to assign higher diagnostic probability to those supported by the familiar symptoms, this effect was significantly larger for participants that engaged in active retrieval compared to passive rehearsal. There were also significant differences in performance across the given diagnoses, potentially due to differences in established knowledge of the disorders. To test this prediction, Experiment 2 compared performance on the described experiment between a participant group that received the standard diagnostic labels to a group that received fictional diagnostic labels, nonsense words designed to remove prior knowledge with each diagnosis. As predicted, there was no effect of diagnosis on task performance for the fictional label group. These results provide new insight on the impact of learning strategy and prior knowledge in fostering transfer of learning, potentially contributing to expert development in medicine.

An Ecological Account of Clinical Reasoning

PURPOSE

The prevailing paradigms of clinical reasoning conceptualize context either as noise that masks, or as external factors that influence, the internal cognitive processes involved in reasoning. The authors reimagined clinical reasoning through the lens of ecological psychology to enable new ways of understanding context-specific manifestations of clinical performance and expertise, and the bidirectional ways in which individuals and their environments interact.

METHOD

The authors performed a critical review of foundational and current literature from the field of ecological psychology to explore the concepts of clinical reasoning and context as presented in the health professions education literature.

RESULTS

Ecological psychology offers several concepts to explore the relationship between an individual and their context, including affordance, effectivity, environment, and niche. Clinical reasoning may be framed as an emergent phenomenon of the interactions between a clinician's effectivities and the affordances in the clinical environment. Practice niches are the outcomes of historical efforts to optimize practice and are both specialty-specific and geographically diverse.

CONCLUSIONS

In this framework, context specificity may be understood as fundamental to clinical reasoning. This changes the authors' understanding of expertise, expert decision making, and definition of clinical error, as they depend on both the expert's actions and the context in which they acted. Training models incorporating effectivities and affordances might allow for antiableist formulations of competence that apply learners' abilities to solving problems in context. This could offer both new means of training and improve access to training for learners of varying abilities. Rural training programs and distance education can leverage technology to provide comparable experience to remote audiences but may benefit from additional efforts to integrate learners into local practice niches.

How to support dental students in reading radiographs: effects of a gaze-based compare-and-contrast intervention

In dental medicine, interpreting radiographs (i.e., orthopantomograms, OPTs) is an error-prone process, even in experts. Effective intervention methods are therefore needed to support students in improving their image reading skills for OPTs. To this end, we developed a compare-and-contrast intervention, which aimed at supporting students in achieving full coverage when visually inspecting OPTs and, consequently, obtaining a better diagnostic performance. The comparison entailed a static eye movement visualization (heat map) on an OPT showing full gaze coverage from a peer-model (other student) and another heat map showing a student's own gaze behavior. The intervention group (N = 38) compared five such heat map combinations, whereas the control group (N = 23) diagnosed five OPTs. Prior to the experimental variation (pre-test) and after it (post-test), students in both conditions searched for anomalies in OPTs while their gaze was recorded. Results showed that students in the intervention group covered more areas of the OPTs and looked less often and for a shorter amount of time at anomalies after the intervention. Furthermore, they fixated on low-prevalence anomalies earlier and high-prevalence anomalies later during the inspection. However, the students in the intervention group did not show any meaningful improvement in detection rate and made more false positive errors compared to the control group. Thus, the intervention guided visual attention but did not improve diagnostic performance substantially. Exploratory analyses indicated that further interventions should teach knowledge about anomalies rather than focusing on full coverage of radiographs.

Aha! Taking on the myth that simulation-derived surprise enhances learning

OBJECTIVES

This paper aims to discuss the recurring education-related issue of the high-fidelity simulation myth. In the current instantiation, educators erroneously believe that trainees benefit from authentic uncertainty and surprise in simulation-based training.

METHODS

We explore the origins of this myth within the experiential learning and social constructivism theories and propose an evidence-based solution of transparent and guided instruction in simulation.

RESULTS

Constructivist theories highlight meaning making as the benefit of inquiry and discovery learning strategies. Inappropriate translation of this epistemology into an element of curriculum design creates unfortunate unintended consequences.

CONCLUSIONS

We propose that the translation of constructivist theories of learning within simulation-based education has resulted in a pervasive myth, which decrees that scenarios must introduce realistic tension or surprises to encourage exploration and insightful problem solving. We argue that this myth is masquerading as experiential learning. In this narrative review, we interpret our experiences and observations of simulation-based education through our expertise in education science and curriculum design. We offer anecdotal evidence along with a review of selected literature to establish the presence of this previously undetected myth.