Guided Reflection Interventions Show No Effect on Diagnostic Accuracy in Medical Students

Association of diagnostic error education and recognition frequency among Japanese medical students: a nationwide cross-sectional study

OBJECTIVES

Diagnostic errors pose a significant risk to patient safety and have substantial medical and economic consequences. Despite their importance, diagnostic error education is currently lacking in standard pre-graduate curricula. This study aimed to investigate the incidence of diagnostic errors and the frequency of recognition among medical students in Japan.

METHODS

A pilot survey was conducted immediately after the General Medicine In-Training Examination (GM-ITE), a comprehensive post-graduation test, administered to new residents right after graduation from medical school. The survey assessed whether they received education on diagnostic errors during their formal undergraduate medical education and whether they recognized diagnostic errors during their clinical training.

RESULTS

Of the 564 examinees, 421 participated in the study. The majority of participants (63.9 %) reported receiving education on diagnostic errors, and 15.7 % recognized diagnostic errors during their clinical training. Significantly, those who received education on diagnostic errors had a higher rate of recognizing such errors compared to those who did not (19.7 vs. 8.6 %; p=0.0017).

CONCLUSIONS

These findings suggest that the recognition rate of diagnostic errors increases with improved literacy in diagnostic error education. This highlights the importance of incorporating diagnostic error education into medical curricula to develop effective strategies to prevent and manage diagnostic errors, and thereby enhance medical and patient safety. However, this study did not examine the specific educational content of the errors or the details of the recognition, necessitating further investigation in the future.

Reflecting on diagnosis: The Metacognitive Diagnostic Reasoning Model©

BACKGROUND

Diagnostic reasoning is a complex cognitive process that requires intuitive, heuristic processing from knowledge and experience, as well as deliberate and reflective thinking. Evidence on interventions to improve diagnostic reasoning is inconsistent, in part because different terms and models are used to guide research.

PURPOSE

To present a model of the factors of diagnostic reasoning in Advanced Practice Registered Nurses (APRNs), based on a review of the literature supporting the Metacognitive Diagnostic Reasoning (MDR) Model©.

METHODOLOGY

A review of the literature through systematic database search, historical articles, texts, and documents was conducted from inception through August 2023 (N = 41).

RESULTS

The MDR model depicts the following concepts: (1) metacognition is viewed as the driving force of diagnostic reasoning; (2) dual cognitive processing and knowledge representations are used; (3) once a diagnosis, treatment plan, and evaluation are complete, APRNs use metacognition for the next patient encounter, using repetition for improvement of the process.

CONCLUSIONS

The model combines key concepts of diagnostic reasoning, providing a framework for researchers to develop and test, and faculty to teach and evaluate this complex process in learners.

IMPLICATIONS

The MDR model provides research opportunities to validate its usefulness and effectiveness and a framework to guide educational research, contributing to the body of evidence-based practice in nursing. In addition, it may foster ongoing education and training that could lead to more accurate diagnoses and treatment plans, ultimately improving patient care by reducing diagnostic errors.

Diagnostic Reasoning Competency and Accuracy by Nurse Practitioner Students Following the Use of Structured Reflection in Simulation: A Mixed-Methods Experiment

AIM

The aim of this study was to examine the effect of structured reflection used during a simulated patient's diagnostic workup on diagnostic reasoning competency and accuracy and explore participants' cognitive bias experience and perceived utility of structured reflection.

BACKGROUND

Reasoning flaws may lead to diagnostic errors. Medical learners who used structured reflection demonstrated improved diagnosis accuracy.

METHOD

Embedded mixed-methods experiment examined diagnostic reasoning competency and accuracy of nurse practitioner students who did and did not use structured reflection. Cognitive bias experience and perceptions of structured reflection's utility were explored.

RESULTS

Diagnostic Reasoning Assessment mean competency scores and categories were not changed. Accuracy trended toward improvement with structured reflection. The theme, diagnostic verification, prompted diagnosis change by both structured reflection users and control participants.

CONCLUSION

Despite no changes in quantitative outcomes, explicit users of structured reflection believed that this strategy is helpful to their reasoning, and control participants used the strategy's components with the same noted benefits.

Effect on diagnostic accuracy of cognitive reasoning tools for the workplace setting: systematic review and meta-analysis

BACKGROUND

Preventable diagnostic errors are a large burden on healthcare. Cognitive reasoning tools, that is, tools that aim to improve clinical reasoning, are commonly suggested interventions. However, quantitative estimates of tool effectiveness have been aggregated over both workplace-oriented and educational-oriented tools, leaving the impact of workplace-oriented cognitive reasoning tools alone unclear. This systematic review and meta-analysis aims to estimate the effect of cognitive reasoning tools on improving diagnostic performance among medical professionals and students, and to identify factors associated with larger improvements.

METHODS

Controlled experimental studies that assessed whether cognitive reasoning tools improved the diagnostic accuracy of individual medical students or professionals in a workplace setting were included. Embase.com, Medline ALL via Ovid, Web of Science Core Collection, Cochrane Central Register of Controlled Trials and Google Scholar were searched from inception to 15 October 2021, supplemented with handsearching. Meta-analysis was performed using a random-effects model.

RESULTS

The literature search resulted in 4546 articles of which 29 studies with data from 2732 participants were included for meta-analysis. The pooled estimate showed considerable heterogeneity (I2=70%). This was reduced to I2=38% by removing three studies that offered training with the tool before the intervention effect was measured. After removing these studies, the pooled estimate indicated that cognitive reasoning tools led to a small improvement in diagnostic accuracy (Hedges' g=0.20, 95% CI 0.10 to 0.29, p<0.001). There were no significant subgroup differences.

CONCLUSION

Cognitive reasoning tools resulted in small but clinically important improvements in diagnostic accuracy in medical students and professionals, although no factors could be distinguished that resulted in larger improvements. Cognitive reasoning tools could be routinely implemented to improve diagnosis in practice, but going forward, more large-scale studies and evaluations of these tools in practice are needed to determine how these tools can be effectively implemented.

PROSPERO REGISTRATION NUMBER

CRD42020186994.

The reduction of race and gender bias in clinical treatment recommendations using clinician peer networks in an experimental setting

Bias in clinical practice, in particular in relation to race and gender, is a persistent cause of healthcare disparities. We investigated the potential of a peer-network approach to reduce bias in medical treatment decisions within an experimental setting. We created "egalitarian" information exchange networks among practicing clinicians who provided recommendations for the clinical management of patient scenarios, presented via standardized patient videos of actors portraying patients with cardiac chest pain. The videos, which were standardized for relevant clinical factors, presented either a white male actor or Black female actor of similar age, wearing the same attire and in the same clinical setting, portraying a patient with clinically significant chest pain symptoms. We found significant disparities in the treatment recommendations given to the white male patient-actor and Black female patient-actor, which when translated into real clinical scenarios would result in the Black female patient being significantly more likely to receive unsafe undertreatment, rather than the guideline-recommended treatment. In the experimental control group, clinicians who were asked to independently reflect on the standardized patient videos did not show any significant reduction in bias. However, clinicians who exchanged real-time information in structured peer networks significantly improved their clinical accuracy and showed no bias in their final recommendations. The findings indicate that clinician network interventions might be used in healthcare settings to reduce significant disparities in patient treatment.

Differential diagnosis checklists reduce diagnostic error differentially: A randomised experiment

INTRODUCTION

Wrong and missed diagnoses contribute substantially to medical error. Can a prompt to generate alternative diagnoses (prompt) or a differential diagnosis checklist (DDXC) increase diagnostic accuracy? How do these interventions affect the diagnostic process and self-monitoring?

METHODS

Advanced medical students (N = 90) were randomly assigned to one of four conditions to complete six computer-based patient cases: group 1 (prompt) was instructed to write down all diagnoses they considered while acquiring diagnostic test results and to finally rank them. Groups 2 and 3 received the same instruction plus a list of 17 differential diagnoses for the chief complaint of the patient. For half of the cases, the DDXC contained the correct diagnosis (DDXC+), and for the other half, it did not (DDXC-; counterbalanced). Group 4 (control) was only instructed to indicate their final diagnosis. Mixed-effects models were used to analyse results.

RESULTS

Students using a DDXC that contained the correct diagnosis had better diagnostic accuracy, mean (standard deviation), 0.75 (0.44), compared to controls without a checklist, 0.49 (0.50), P < 0.001, but those using a DDXC that did not contain the correct diagnosis did slightly worse, 0.43 (0.50), P = 0.602. The number and relevance of diagnostic tests acquired were not affected by condition, nor was self-monitoring. However, participants spent more time on a case in the DDXC-, 4:20 min (2:36), P ≤ 0.001, and DDXC+ condition, 3:52 min (2:09), than in the control condition, 2:59 min (1:44), P ≤ 0.001.

DISCUSSION

Being provided a list of possible diagnoses improves diagnostic accuracy compared with a prompt to create a differential diagnosis list, if the provided list contains the correct diagnosis. However, being provided a diagnosis list without the correct diagnosis did not improve and might have slightly reduced diagnostic accuracy. Interventions neither affected information gathering nor self-monitoring.

Methods to Improve Diagnostic Reasoning in Undergraduate Medical Education in the Clinical Setting: a Systematic Review

BACKGROUND

In recent years, there have been numerous studies exploring different teaching methods for improving diagnostic reasoning in undergraduate medical students. This systematic review examines and summarizes the evidence for the effectiveness of these teaching methods during clinical training.

METHODS

PubMed, Embase, Scopus, and ERIC were searched. The inclusion criteria for the review consist of articles describing (1) methods to enhance diagnostic reasoning, (2) in a clinical setting (3) on medical students. Articles describing original research using qualitative, quantitative, or mixed study designs and published within the last 10 years (1 April 2009-2019) were included. Results were screened and evaluated for eligibility. Relevant data were then extracted from the studies that met the inclusion criteria.

RESULTS

Sixty-seven full-text articles were first identified. Seventeen articles were included in this review. There were 13 randomized controlled studies and four quasi-experimental studies. Of the randomized controlled studies, six discussed structured reflection, four self-explanation, and three prompts for generating differential diagnoses. Of the remaining four studies, two employed the SNAPPS¹ technique for case presentation. Two other studies explored schema-based instruction and using illness scripts. Twelve out of 17 studies reported improvement in clinical reasoning after the intervention. All studies ranked level two on the New World Kirkpatrick model.

DISCUSSION

The authors posit a framework to teach diagnostic reasoning in the clinical setting. The framework targets specific deficiencies in the students' reasoning process. There remains a lack of studies comparing the effectiveness of different methods. More comparative studies with standardized assessment and evaluation of long-term effectiveness of these methods are recommended.

A diagnostic time-out to improve differential diagnosis in pediatric abdominal pain

BACKGROUND

Pediatric abdominal pain is challenging to diagnose and often results in unscheduled return visits to the emergency department. External pressures and diagnostic momentum can impair physicians from thoughtful reflection on the differential diagnosis (DDx). We implemented a diagnostic time-out intervention and created a scoring tool to improve the quality and documentation rates of DDx. The specific aim of this quality improvement (QI) project was to increase the frequency of resident and attending physician documentation of DDx in pediatric patients admitted with abdominal pain by 25% over 6 months.

METHODS

We reviewed a total of 165 patients admitted to the general pediatrics service at one institution. Sixty-four history and physical (H&P) notes were reviewed during the baseline period, July-December 2017; 101 charts were reviewed post-intervention, January-June 2018. Medical teams were tasked to perform a diagnostic time-out on all patients during the study period. Metrics tracked monthly included percentage of H&Ps with a 'complete' DDx and quality scores (Qs) using our Differential Diagnosis Scoring Rubric.

RESULTS

At baseline, 43 (67%) resident notes and 49 (77%) attending notes documented a 'complete' DDx. Post-intervention, 59 (58%) resident notes and 69 (68%) attending notes met this criteria. Mean Qs, pre- to post-intervention, for resident-documented differential diagnoses increased slightly (2.41-2.47, p = 0.73), but attending-documented DDx did not improve (2.85-2.82, p = 0.88).

CONCLUSIONS

We demonstrated a marginal improvement in the quality of resident-documented DDx. Expansion of diagnoses considered within a DDx may contribute to higher diagnostic accuracy.

Education initiatives in cognitive debiasing to improve diagnostic accuracy in student providers: A scoping review

BACKGROUND

The high prevalence of diagnostic errors by health care providers has prompted medical educators to examine cognitive biases and debiasing strategies in an effort to prevent these errors. The National Academy of Medicine hypothesized that explicit diagnostic reasoning education of all health care professionals can improve diagnostic accuracy.

OBJECTIVES

The purpose of this scoping review is to identify, analyze, and summarize the existing literature on student health care providers' use of cognitive debiasing strategies to reduce diagnostic error.

DATA SOURCES

The review was guided by the Joanna Briggs Institute methodology for scoping reviews. A systematic search of PubMed, CINAHL, PsychINFO, and Scopus databases for debiasing strategies in student provider education yielded 33 studies.

CONCLUSIONS

The 33 studies included in this review represent four categories of debiasing strategies: increased medical knowledge or experience (seven studies), guided reflection (eight studies), self-explanation of reasoning (nine studies), and checklists to expand diagnosis considerations (seven studies). The studies were inclusive of medical students and residents; no studies included nurse practitioner (NP) students. Guided reflection, the most clearly defined and implemented strategy, showed the most promise for improvement of diagnostic accuracy. Educational interventions were wide ranging in content and delivery but did yield a path for future research.

IMPLICATIONS FOR PRACTICE

There are myriad debiasing strategies student providers may use to mitigate cognitive bias. Structured reflection and education initiatives demonstrated the most consistent improvements in diagnostic accuracy. Future studies on debiasing strategies must include NP students to understand their response to these initiatives.