Dual processing theory and experts' reasoning: exploring thinking on national multiple-choice questions

Beyond thinking fast and slow: Implications of a transtheoretical model of clinical reasoning and error on teaching, assessment, and research

From dual process to a family of theories known collectively as situativity, both micro and macro theories of cognition inform our current understanding of clinical reasoning (CR) and error. CR is a complex process that occurs in a complex environment, and a nuanced, expansive, integrated model of these theories is necessary to fully understand how CR is performed in the present day and in the future. In this perspective, we present these individual theories along with figures and descriptive cases for purposes of comparison before exploring the implications of a transtheoretical model of these theories for teaching, assessment, and research in CR and error.

The impact of patient skin colour on diagnostic ability and confidence of medical students

Previous literature has explored unconscious racial biases in clinical education and medicine, finding that people with darker skin tones can be underrepresented in learning resources and managed differently in a clinical setting. This study aimed to examine whether patient skin colour can affect the diagnostic ability and confidence of medical students, and their cognitive reasoning processes. We presented students with 12 different clinical presentations on both white skin (WS) and non-white skin (NWS). A think aloud (TA) study was conducted to explore students' cognitive reasoning processes (n = 8). An online quiz was also conducted where students submitted a diagnosis and confidence level for each clinical presentation (n = 185). In the TA interviews, students used similar levels of information gathering and analytical reasoning for each skin type but appeared to display increased uncertainty and reduced non-analytical reasoning methods for the NWS images compared to the WS images. In the online quiz, students were significantly more likely to accurately diagnose five of the 12 clinical presentations (shingles, cellulitis, Lyme disease, eczema and meningococcal disease) on WS compared to NWS (p < 0.01). With regards to students' confidence, they were significantly more confident diagnosing eight of the 12 clinical presentations (shingles, cellulitis, Lyme disease, eczema, meningococcal disease, urticaria, chickenpox and Kawasaki disease) on WS when compared to NWS (p < 0.01). These findings highlight the need to improve teaching resources to include a greater diversity of skin colours exhibiting clinical signs, to improve students' knowledge and confidence, and ultimately, to avoid patients being misdiagnosed due to the colour of their skin.

Clinical reasoning pattern used in oral health problem solving - A case study in Indonesian undergraduate dental students

BACKGROUND

Health professionals are known to use various combinations of knowledge and skills, such as critical thinking, clinical reasoning, clinical judgment, problem-solving, and decision-making, in conducting clinical practice. Clinical reasoning development is influenced by knowledge and experience, the more knowledge and experience, the more sophisticated clinical reasoning will be. However, clinical reasoning research in dentistry shows varying results .

AIMS

This study aims to observe the clinical reasoning pattern of undergraduate dental students when solving oral health problems, and their accordance with their knowledge acquisition.

MATERIAL AND METHODS

This qualitative study employed the think-aloud method and the result was assessed through verbal protocol analyses. Five respondents from final year dental undergraduate students were agreed to participate. A unique hypothetical clinical scenario was used as a trigger. The audio data were transcribed, interpreted, and categorized as a clinical reasoning pattern; and the concept maps created were assessed by a Structure of Learning Outcomes (SOLO) taxonomy as knowledge acquisition.

RESULTS

Observations on clinical reasoning patterns and the level of knowledge acquisition in five undergraduate dental students showed varying results. They applied clinical reasoning patterns according to their knowledge acquisition during didactical phase. Learners with inadequate knowledge relied on guessing, meanwhile learners with adequate knowledge applied more sophisticated reasoning pattern when solving problems.

CONCLUSIONS

Various problem-solving strategies were encountered in this study, which corresponded to the level of knowledge acquisition. Dental institutions must set minimum standards regarding the acquisition of conceptual knowledge accompanied by improvement of clinical reasoning skills, as well as refinement of knowledge and procedural skills.

Types of clinical reasoning in a summative clerkship oral examination

INTRODUCTION

Dual-process theory characterizes clinical reasoning (CR) as Type 1 (intuitive) and Type 2 (analytical) thinking. This study examined CR on a summative clinical clerkship structured clinical oral examination (SCOE).

METHODS AND SUBJECTS

511 clinical clerks at the University of Toronto underwent SCOEs. Type 1, Type 2, and Global CR performance were compared to other internal medicine clerkship assessments using descriptive statistics and Spearman correlations.

RESULTS

Clinical clerks achieved mean marks >75% on the three clinical reasoning stations, on Type 1 and 2 CR tasks, and the overall SCOE. Performance on the SCOE CR stations correlated with each of the other clerkship assessments: written examination, inpatient, and ambulatory clinic assessments. The correlation of performance between Type 1 and Type 2 clinical reasoning tasks was statistically significant but weak (r_s = 0.28). This suggests that defined measures of Type 1 and Type 2 reasoning were indeed assessing distinct constructs.

CONCLUSION

Clinical clerks used both Type 1 and Type 2 reasoning with success. This study's characterization of Type 1 and Type 2 CR as separate domains, distinct from existing measures on the SCOE as well as the other clerkship assessments, can suggest a further addition to multimodal clerkship assessment.

Patterns of Domain-Specific Learning Among Medical Undergraduate Students in Relation to Confidence in Their Physiology Knowledge: Insights From a Pre-post Study

RESEARCH FOCUS

The promotion of domain-specific knowledge is a central goal of higher education and, in the field of medicine, it is particularly essential to promote global health. Domain-specific knowledge on its own is not exhaustive; confidence regarding the factual truth of this knowledge content is also required. An increase in both knowledge and confidence is considered a necessary prerequisite for making professional decisions in the clinical context. Especially the knowledge of human physiology is fundamental and simultaneously critical to medical decision-making. However, numerous studies have shown difficulties in understanding and misconceptions in this area of knowledge. Therefore, we investigate (i) how preclinical medical students acquire knowledge in physiology over the course of their studies and simultaneously gain confidence in the correctness of this knowledge as well as (ii) the interrelations between these variables, and (iii) how they affect the development of domain-specific knowledge.

METHOD

In a pre-post study, 169 medical students' development of physiology knowledge and their confidence related to this knowledge were assessed via paper-pencil questionnaires before and after attending physiology seminars for one semester. Data from a longitudinal sample of n = 97 students were analyzed using mean comparisons, regression analyses, and latent class analyses (LCAs). In addition, four types of item responses were formed based on confidence and correctness in the knowledge test.

RESULTS

We found a significant and large increase in the students' physiology knowledge, with task-related confidence being the strongest predictor (apart from learning motivation). Moreover, a significantly higher level of confidence at t2 was confirmed, with the level of prior confidence being a strong predictor (apart from knowledge at t2). Furthermore, based on the students' development of knowledge and confidence levels between measurement points, three empirically distinct groups were distinguished: knowledge gainers, confidence gainers, and overall gainers. The students whose confidence in incorrect knowledge increased constituted one particularly striking group. Therefore, the training of both knowledge and the ability to critically reflect on one's knowledge and skills as well as an assessment of their development in education is required, especially in professions such as medicine, where knowledge-based decisions made with confidence are of vital importance.

Thinking differently - Students' cognitive processes when answering two different formats of written question

BACKGROUND

Single-best answer questions (SBAQs) are common but are susceptible to cueing. Very short answer questions (VSAQs) could be an alternative, and we sought to determine if students' cognitive processes varied across question types and whether students with different performance levels used different methods for answering questions.

METHODS

We undertook a 'think aloud' study, interviewing 21 final year medical students at five UK medical schools. Each student described their thought processes and methods used for eight questions of each type. Responses were coded and quantified to determine the relative frequency with which each method was used, denominated on the number of times a method could have been used.

RESULTS

Students were more likely to use analytical reasoning methods (specifically identifying key features) when answering VSAQs. The use of test-taking behaviours was more common for SBAQs; students frequently used the answer options to help them reach an answer. Students acknowledged uncertainty more frequently when answering VSAQs. Analytical reasoning was more commonly used by high-performing students compared with low-performing students.

CONCLUSIONS

Our results suggest that VSAQs encourage more authentic clinical reasoning strategies. Differences in cognitive approaches used highlight the need for focused approaches to teaching clinical reasoning and dealing with uncertainty.

PHYSICIAN CHARACTERISTICS THAT INFLUENCE PATIENT PARTICIPATION IN THE TREATMENT OF PRIMARY IMMUNODEFICIENCY

OBJECTIVE

Shared decision making (SDM) is recommended to improve healthcare quality. Physicians who use a rational decision-making style and patient-centric approach are more likely to incorporate SDM into clinical practice. This paper explores how certain physician characteristics such as gender, age, race, experience, and specialty explain patient participation.

METHODS

A multi-group structural equation model tested the relationship between physician decision-making styles, patient-centered care, physician characteristics, and patient participation in clinical treatment decisions. A survey was completed by 330 physicians who treat primary immunodeficiency. Sample group responses were compared between groups across specialty, age, race, experience, or gender.

RESULTS

A patient-centric approach was the main factor that encouraged SDM independent of physician decision-making style with both treatment protocols and product choices. The positive effect of patient-centrism is stronger for immunologists, more experienced physicians, or male physicians. A rational decision-making style increases participation for non-immunologists, older physicians, white physicians, less-experienced physicians and female physicians.

CONCLUSION

A patient-centric approach, rational decision-making and certain physician characteristics help explain patient participation in clinical decisions. Practice Implications Future SDM research and policy initiatives should focus on physician adoption of patient-centric approaches to chronic care diseases and the potential bias associated with physician characteristics and decision-making style.

Using Intervention Mapping to Develop a Digital Self-Management Program for People With Type 2 Diabetes: Tutorial on MyDESMOND

Digital health interventions (DHIs) are increasingly becoming integrated into diabetes self-management to improve behavior. Despite DHIs becoming available to people with chronic conditions, the development strategies and processes undertaken are often not well described. With theoretical frameworks available in current literature, it is vital that DHIs follow a shared language and communicate a robust development process in a comprehensive way. This paper aims to bring a unique perspective to digital development, as it describes the systematic process of developing a digital self-management program for people with type 2 diabetes, MyDESMOND. We provide a step-by-step guide, based on the intervention mapping (IM) framework to illustrate the process of adapting an existing face-to-face self-management program (diabetes education and self- management for ongoing and newly diagnosed, DESMOND) and translating it to a digital platform (MyDESMOND). Overall, this paper describes the 4 IM steps that were followed to develop MyDESMOND—step 1 to establish a planning group and a patient and public involvement group to describe the context of the intervention and program goals, step 2 to identify objectives and determinants at early design stages to maintain a focus on the strategies adopted, step 3 to generate the program components underpinned by appropriate psychological theories and models, and step 4 to develop the program content and describe the iterative process of refining the content and format of the digital program for implementation. This paper concludes with a number of key learnings collated throughout our development process, which we hope other researchers may find useful when developing DHIs for chronic conditions.

The Effect of Simulation Wars on EM Residents' Global Rating Scales and In-Training Examination Scores

Purpose

The Accreditation Council for Graduate Medical Education (ACGME) restricted the duty hours for residents training in 2004. With less time to learn medicine, residents may not develop the clinical reasoning skills needed. Simulation can provide a remedy for this lack of time by allowing residents to practice skills and develop clinical reasoning in a simulated environment. Simulation Wars (SimWars), a clinical reasoning simulation has been shown to improve clinical reasoning skills. The purpose of the study was to investigate the effect of Simulation Wars on In-Training Examination (ITE) Scores and Global Rating Scale (GRS) Scores in Emergency Medicine (EM) residents.

Methods

The Quasi-Experimental design was used in this retrospective study. The main comparison was between historical controls, the residents who did not participate in the Simulation Wars, and the intervention group comprised of residents who participated in the SimWars.

Results

A total of 127 residents were participants in this study including 70 from the intervention and 57 from the historical control group. There were no significant differences found in GRS scores between both groups except for Communication and Professionalism (p<0.001). No overall improvement in ITE scores for the control group and for the intervention group was found. Furthermore, within the intervention group, while comparing those residents who participated in certain subcategories of SimWars and those who did not, there was a significant improvement in ITE scores in the subcategories of Thoracic Disorders, Abdominal/Gastrointestinal, Trauma Disorders and OBGyn.

Conclusion

SimWars in the subcategories of Abdominal/Gastrointestinal, Thoracic, OBGyn and Trauma were found to be associated with improved ITE scores in those subcategories. Since Emergency Medicine utilizes extensive clinical reasoning skills, SimWars may provide better educational opportunities for EM residents.

The neurocognitive gains of diagnostic reasoning training using simulated interactive veterinary cases

The present longitudinal study ascertained training-associated transformations in the neural underpinnings of diagnostic reasoning, using a simulation game named “Equine Virtual Farm” (EVF). Twenty participants underwent structural, EVF/task-based and resting-state MRI and diffusion tensor imaging (DTI) before and after completing their training on diagnosing simulated veterinary cases. Comparing playing veterinarian versus seeing a colorful image across training sessions revealed the transition of brain activity from scientific creativity regions pre-training (left middle frontal and temporal gyrus) to insight problem-solving regions post-training (right cerebellum, middle cingulate and medial superior gyrus and left postcentral gyrus). Further, applying linear mixed-effects modelling on graph centrality metrics revealed the central roles of the creative semantic (inferior frontal, middle frontal and angular gyrus and parahippocampus) and reward systems (orbital gyrus, nucleus accumbens and putamen) in driving pre-training diagnostic reasoning; whereas, regions implicated in inductive reasoning (superior temporal and medial postcentral gyrus and parahippocampus) were the main post-training hubs. Lastly, resting-state and DTI analysis revealed post-training effects within the occipitotemporal semantic processing region. Altogether, these results suggest that simulation-based training transforms diagnostic reasoning in novices from regions implicated in creative semantic processing to regions implicated in improvised rule-based problem-solving.

Shared decision making: Does a physician's decision-making style affect patient participation in treatment choices for primary immunodeficiency?

Overall health care spending in the United States is equivalent to more than 15% of GDP, yet outcomes rank below the top 25 in most quality categories when compared with other Organization for Economic Cooperation and Development (OECD) countries. The majority of spending is consumed by small patient populations with chronic diseases. Experts believe increased patient-physician shared decision making (SDM) should result in better overall longitudinal care but understanding the physician's role in facilitating SDM is limited. Structural equation modelling was applied to results of a 2016 questionnaire-based survey of 330 US physicians who treat approximately 55% of primary immune deficiency requiring immune globulin therapy; it tested the relationship between slow/rational vs fast/intuitive decision-making styles and SDM as mediated by patient-centric care and moderated by physician's trust in the patient. The results showed a statistically significant relationship between slow/rational decision making and SDM. The results also suggest differences related to age, gender, education, and race but no differences related to trust.

UK vs US physician decision-making in the treatment of haemophilia

INTRODUCTION

Patient-physician shared decision-making (SDM) has become increasingly seen as having a positive effect on management of chronic diseases. However, little is known of the factors that encourage SDM or how effective it may be at improving health outcomes or how cost-effective it is.

AIM

To investigate the uses and applications of patient physician-SDM in the management of haemophilia and the influence of healthcare systems in the United States and the United Kingdom.

METHODS

This was a qualitative study based on interviews with treatment experts in the United States and United Kingdom. A grounded theory approach was used to analyse the data from the transcribed interviews and themes that emerged as related to the decision influencers. Twelve physicians from each country were interviewed by the author.

RESULTS

Treatment guidelines were viewed as having only limited applicability because of the lack of universal best options in haemophilia. The US physicians in the sample appeared to be more influenced by patient preferences than physicians in the UK, who instead tended to follow policies and standards of care more closely. Physicians in both countries commented that many of their patents had become highly knowledgeable of their bleeding disorder. US physicians were sometimes limited by insurance company policies but also reported that they were often successful in appealing insurance decisions.

CONCLUSION

The research suggests that there are different influences on decision-making between healthcare systems; patients and overarching healthcare systems play a major role in how physicians treat haemophilia.

Perception of the usability and implementation of a metacognitive mnemonic to check cognitive errors in clinical setting

BACKGROUND

Establishing a diagnosis is a complex, iterative process involving patient data gathering, integration and interpretation. Premature closure is a fallacious cognitive tendency of closing the diagnostic process before sufficient data have been gathered. A proposed strategy to minimize premature closure is the use of a checklist to trigger metacognition (the process of monitoring one's own thinking). A number of studies have suggested the effectiveness of this strategy in classroom settings. This qualitative study examined the perception of usability of a metacognitive mnemonic checklist called TWED checklist (where the letter "T = Threat", "W = What if I am wrong? What else?", "E = Evidence" and "D = Dispositional influence") in a real clinical setting.

METHOD

Two categories of participants, i.e., medical doctors (n = 11) and final year medical students (Group 1, n = 5; Group 2, n = 10) participated in four separate focus group discussions. Nielsen's 5 dimensions of usability (i.e. learnability, effectiveness, memorability, errors, and satisfaction) and Pentland's narrative network were adapted as the framework to study the usability and the implementation of the checklist in a real clinical setting respectively.

RESULTS

Both categories (medical doctors and medical students) of participants found that the TWED checklist was easy to learn and effective in promoting metacognition. For medical student participants, items "T" and "W" were believed to be the two most useful aspects of the checklist, whereas for the doctor participants, it was item "D". Regarding its implementation, item "T" was applied iteratively, items "W" and "E" were applied when the outcomes did not turn out as expected, and item "D" was applied infrequently. The one checkpoint where all four items were applied was after the initial history taking and physical examination had been performed to generate the initial clinical impression.

CONCLUSION

A metacognitive checklist aimed to check cognitive errors may be a useful tool that can be implemented in the real clinical setting.

A mixed-methods exploration of cognitive dispositions to respond and clinical reasoning errors with multiple choice questions

BACKGROUND

Cognitive dispositions to respond (i.e., cognitive biases and heuristics) are well-established clinical reasoning phenomena. While thought by many to be error-prone, some scholars contest that these cognitive dispositions to respond are pragmatic solutions for reasoning through clinical complexity that are associated with errors largely due to hindsight bias and flawed experimental design. The purpose of this study was to prospectively identify cognitive dispositions to respond occurring during clinical reasoning to determine whether they are actually associated with increased odds of an incorrect answer (i.e., error).

METHODS

Using the cognitive disposition to respond framework, this mixed-methods study applied a constant comparative qualitative thematic analysis to transcripts of think alouds performed during completion of clinical-vignette multiple-choice questions. The number and type of cognitive dispositions to respond associated with both correct and incorrect answers were identified. Participants included medical students, residents, and attending physicians recruited using maximum variation strategies. Data were analyzed using generalized estimating equations binary logistic model for repeated, within-subjects measures.

RESULTS

Among 14 participants, there were 3 cognitive disposition to respond categories - Cognitive Bias, Flaws in Conceptual Understanding, and Other Vulnerabilities - with 13 themes identified from the think aloud transcripts. The odds of error increased to a statistically significant degree with a greater per-item number of distinct Cognitive Bias themes (OR = 1.729, 95% CI [1.226, 2.437], p = 0.002) and Other Vulnerabilities themes (OR = 2.014, 95% CI [1.280, 2.941], p < 0.001), but not with Flaws in Conceptual Understanding themes (OR = 1.617, 95% CI [0.961, 2.720], p = 0.070).

CONCLUSION

This study supports the theoretical understanding of cognitive dispositions to respond as phenomena associated with errors in a new prospective manner. With further research, these findings may inform teaching, learning, and assessment of clinical reasoning toward a reduction in patient harm due to clinical reasoning errors.

Advancing clinical reasoning in virtual patients - development and application of a conceptual framework

Background: Clinical reasoning is a complex skill students have to acquire during their education. For educators it is difficult to explain their reasoning to students, because it is partly an automatic and unconscious process. Virtual Patients (VPs) are used to support the acquisition of clinical reasoning skills in healthcare education. However, until now it remains unclear which features or settings of VPs optimally foster clinical reasoning. Therefore, our aims were to identify key concepts of the clinical reasoning process in a qualitative approach and draw conclusions on how each concept can be enhanced to advance the learning of clinical reasoning with virtual patients. Methods: We chose a grounded theory approach to identify key categories and concepts of learning clinical reasoning and develop a framework. Throughout this process, the emerging codes were discussed with a panel of interdisciplinary experts. In a second step we applied the framework to virtual patients. Results: Based on the data we identified the core category as the "multifactorial nature of learning clinical reasoning". This category is reflected in the following five main categories: Psychological Theories, Patient-centeredness, Context, Learner-centeredness, and Teaching/Assessment. Each category encompasses between four and six related concepts. Conclusions: With our approach we were able to elaborate how key categories and concepts of clinical reasoning can be applied to virtual patients. This includes aspects such as allowing learners to access a large number of VPs with adaptable levels of complexity and feedback or emphasizing dual processing, errors, and uncertainty.

Exploring examinee behaviours as validity evidence for multiple-choice question examinations

CONTEXT

Clinical-vignette multiple choice question (MCQ) examinations are used widely in medical education. Standardised MCQ examinations are used by licensure and certification bodies to award credentials that are meant to assure stakeholders as to the quality of physicians. Such uses are based on the interpretation of MCQ examination performance as giving meaningful information about the quality of clinical reasoning. There are several assumptions foundational to these interpretations and uses of standardised MCQ examinations. This study explores the implicit assumption that cognitive processes elicited by clinical-vignette MCQ items are like the processes thought to occur with 'real-world' clinical reasoning as theorised by dual-process theory.

METHODS

Fourteen participants (three medical students, five residents and six staff physicians) completed three sets of five timed MCQ items (total 15) from the Medical Knowledge Self-Assessment Program (MKSAP). Upon answering a set of MCQs, each participant completed a retrospective think aloud (TA) protocol. Using constant comparative analysis (CCA) methods sensitised by dual-process theory, we performed a qualitative thematic analysis.

RESULTS

Examinee behaviours fell into three categories: clinical reasoning behaviours, test-taking behaviours and reactions to the MCQ. Consistent with dual-process theory, statements about clinical reasoning behaviours were divided into two sub-categories: analytical reasoning and non-analytical reasoning. Each of these categories included several themes.

CONCLUSIONS

Our study provides some validity evidence that test-takers' descriptions of their cognitive processes during completion of high-quality clinical-vignette MCQs align with processes expected in real-world clinical reasoning. This supports one of the assumptions important for interpretations of MCQ examination scores as meaningful measures of clinical reasoning. Our observations also suggest that MCQs elicit other cognitive processes, including certain test-taking behaviours, that seem 'inauthentic' to real-world clinical reasoning. Further research is needed to explore if similar themes arise in other contexts (e.g. simulated patient encounters) and how observed behaviours relate to performance on MCQ-based assessments.

Development and Evaluation of a Web-Based Dermatology Teaching Tool for Preclinical Medical Students

INTRODUCTION

There is growing interest in, and emphasis on, electronic teaching tools in medicine. Despite relevant testing on the United States Medical Licensing Examination (USMLE), American medical schools offer limited training in skin disorders. Teaching visual topics like dermatology in classroom formats is challenging. We hypothesized that an electronic module would enhance students' dermatology competency.

METHODS

A self-directed, case-based module was created. To test its efficacy, 40 medical students were randomized to have module access (interventional group) or none (conventional group). Learning outcomes were compared using a multiple-choice exam, including questions relevant and irrelevant to the module. Outcomes included proportions of correctly answered module questions (module scores) and nonmodule questions (nonmodule scores). Difference scores were calculated: (module score) - (nonmodule score). Positive values indicated that knowledge of module questions surpassed that of nonmodule questions. If there were a training effect, the interventional group's difference score should exceed that of the conventional group.

RESULTS

The interventional group scored significantly higher than did the conventional group on module questions-75% (interquartile range [IQR], 69-88) versus 50% (IQR, 38-63), p < .001-and nonmodule questions-85% (IQR, 69-92) versus 69% (IQR, 54-77), p = .02. The Hodges-Lehman median difference estimate of the training effect was 13.0 (95% confidence interval, 0.5-25.5).

DISCUSSION

This e-module is effective at enhancing students' competency in dermatology while emphasizing detailed pathophysiology that prepares them for USMLE Step 1. A module-based curriculum may enhance learning in supplement to traditional teaching modalities.

It is time to progress beyond the System 1 versus System 2 dichotomy

None