Good judgments do not require complex cognition

Dual process models of clinical reasoning: The central role of knowledge in diagnostic expertise

RATIONALE

Research on diagnostic reasoning has been conducted for fifty years or more. There is growing consensus that there are two distinct processes involved in human diagnostic reasoning: System 1, a rapid retrieval of possible diagnostic hypotheses, largely automatic and based to a large part on experiential knowledge, and System 2, a slower, analytical, conscious application of formal knowledge to arrive at a diagnostic conclusion. However, within this broad framework, controversy and disagreement abound. In particular, many authors have suggested that the root cause of diagnostic errors is cognitive biases originating in System 1 and propose that educating learners about the types of cognitive biases and their impact on diagnosis would have a major influence on error reduction.

AIMS AND OBJECTIVES

In the present paper, we take issue with these claims.

METHOD

We reviewed the literature to examine the extent to which this theoretical model is supported by the evidence.

RESULTS

We show that evidence derived from fundamental research in human cognition and studies in clinical medicine challenges the basic assumptions of this theory-that errors arise in System 1 processing as a consequence of cognitive biases, and are corrected by slow, deliberative analytical processing. We claim that, to the contrary, errors derive from both System 1 and System 2 reasoning, that they arise from lack of access to the appropriate knowledge, not from errors of processing, and that the two processes are not essential to the process of diagnostic reasoning.

CONCLUSIONS

The two processing modes are better understood as a consequence of the nature of the knowledge retrieved, not as independent processes.

Critical thinking, biases and dual processing: The enduring myth of generalisable skills

CONTEXT

The myth of generalisable thinking skills in medical education is gaining popularity once again. The implications are significant as medical educators decide on how best to use limited resources to prepare trainees for safe medical practice. This myth-busting critical review cautions against the proliferation of curricular interventions based on the acquisition of generalisable skills.

STRUCTURE

This paper begins by examining the recent history of general thinking skills, as defined by research in cognitive psychology and medical education. We describe three distinct epochs: (a) the Renaissance, which marked the beginning of cognitive psychology as a discipline in the 1960s and 1970s and was paralleled by educational reforms in medical education focused on problem solving and problem-based learning; (b) the Enlightenment, when an accumulation of evidence in psychology and in medical education cast doubt on the assumption of general reasoning or problem-solving skill and shifted the focus to consideration of the role of knowledge in expert clinical performance; and (c) the Counter-Enlightenment, in the current time, when the notion of general thinking skills has reappeared under different guises, but the fundamental problems related to lack of generality of skills and centrality of knowledge remain.

CONCLUSIONS

The myth of general thinking skills persists, despite the lack of evidence. Progress in medical education is more likely to arise from devising strategies to improve the breadth and depth of experiential knowledge.

The etiology of diagnostic errors: a controlled trial of system 1 versus system 2 reasoning

PURPOSE

Diagnostic errors are thought to arise from cognitive biases associated with System 1 reasoning, which is rapid and unconscious. The primary hypothesis of this study was that the instruction to be slow and thorough will have no advantage in diagnostic accuracy over the instruction to proceed rapidly.

METHOD

Participants were second-year residents who volunteered after they had taken the Medical Council of Canada (MCC) Qualifying Examination Part II. Participants were tested at three Canadian medical schools (McMaster, Ottawa, and McGill) in 2010 (n = 96) and 2011 (n = 108). The intervention consisted of 20 computer-based internal medicine cases, with instructions either (1) to be as quick as possible but not make mistakes (the Speed cohort, 2010), or (2) to be careful, thorough, and reflective (the Reflect cohort, 2011). The authors examined accuracy scores on the 20 cases, time taken to diagnose cases, and MCC examination performance.

RESULTS

Overall accuracy in the Speed condition was 44.5%, and in the Reflect condition was 45.0%; this was not significant. The Speed cohort took an average of 69 seconds per case versus 89 seconds for the Reflect cohort (P < .001). In both cohorts, cases diagnosed incorrectly took an average of 17 seconds longer than cases diagnosed correctly. Diagnostic accuracy was moderately correlated with performance on both written and problem-solving components of the MCC licensure examination and inversely correlated with time.

CONCLUSIONS

The study demonstrates that simply encouraging slowing down and increasing attention to analytical thinking is insufficient to increase diagnostic accuracy.

The relationship between response time and diagnostic accuracy

PURPOSE

Psychologists theorize that cognitive reasoning involves two distinct processes: System 1, which is rapid, unconscious, and contextual, and System 2, which is slow, logical, and rational. According to the literature, diagnostic errors arise primarily from System 1 reasoning, and therefore they are associated with rapid diagnosis. This study tested whether accuracy is associated with shorter or longer times to diagnosis.

METHOD

Immediately after the 2010 administration of the Medical Council of Canada Qualifying Examination (MCCQE) Part II at three test centers, the authors recruited participants, who read and diagnosed a series of 25 written cases of varying difficulty. The authors computed accuracy and response time (RT) for each case.

RESULTS

Seventy-five Canadian medical graduates (of 95 potential participants) participated. The overall correlation between RT and accuracy was -0.54; accuracy, then, was strongly associated with more rapid RT. This negative relationship with RT held for 23 of 25 cases individually and overall when the authors controlled for participants' knowledge, as judged by their MCCQE Part I and II scores. For 19 of 25 cases, accuracy on each case was positively related to experience with that specific diagnosis. A participant's performance on the test overall was significantly correlated with his or her performance on both the MCCQE Part I and II.

CONCLUSIONS

These results are inconsistent with clinical reasoning models that presume that System 1 reasoning is necessarily more error prone than System 2. These results suggest instead that rapid diagnosis is accurate and relates to other measures of competence.

Using the ACT-R architecture to specify 39 quantitative process models of decision making

Hypotheses about decision processes are often formulated qualitatively and remain silent about the interplay of decision, memorial, and other cognitive processes. At the same time, existing decision models are specified at varying levels of detail, making it difficult to compare them. We provide a methodological primer on how detailed cognitive architectures such as ACT-R allow remedying these problems. To make our point, we address a controversy, namely, whether noncompensatory or compensatory processes better describe how people make decisions from the accessibility of memories. We specify 39 models of accessibility-based decision processes in ACT-R, including the noncompensatory recognition heuristic and various other popular noncompensatory and compensatory decision models. Additionally, to illustrate how such models can be tested, we conduct a model comparison, fitting the models to one experiment and letting them generalize to another. Behavioral data are best accounted for by race models. These race models embody the noncompensatory recognition heuristic and compensatory models as a race between competing processes, dissolving the dichotomy between existing decision models.

Recognition-based judgments and decisions: What we have learned (so far)

This special issue on recognition processes in inferential decision making represents an adversarial collaboration among the three guest editors. This introductory article to the special issue’s third and final part comes in three sections. In Section 1, we summarize the six papers that appear in this part. In Section 2, we give a wrap-up of the lessons learned. Specifically, we discuss (i) why studying the recognition heuristic has led to so much controversy, making it difficult to settle on mutually accepted empirically grounded assumptions, (ii) whether the development of the recognition heuristic and its theoretical descriptions could explain some of the past controversies and misconceptions, (iii) how additional cue knowledge about unrecognized objects could enter the decision process, (iv) why recognition heuristic theory should be complemented by a probabilistic model of strategy selection, and (v) how recognition information might be related to other information, especially when considering real-world applications. In Section 3, we present an outlook on the thorny but fruitful road to cumulative theory integration. Future research on recognition-based inferences should (i) converge on overcoming past controversies, taking an integrative approach to theory building, and considering theories and findings from neighboring fields (such as marketing science and artificial intelligence), (ii) build detailed computational process models of decision strategies, grounded in cognitive architectures, (iii) test existing models of such strategies competitively, (iv) design computational models of the mechanisms of strategy selection, and (v) effectively extend its scope to decision making in the wild, outside controlled laboratory situations.

Forecasting elections with mere recognition from small, lousy samples: A comparison of collective recognition, wisdom of crowds, and representative polls

We investigated the extent to which the human capacity for recognition helps to forecast political elections: We compared naïve recognition-based election forecasts computed from convenience samples of citizens’ recognition of party names to (i) standard polling forecasts computed from representative samples of citizens’ voting intentions, and to (ii) simple—and typically very accurate—wisdom-of-crowds-forecasts computed from the same convenience samples of citizens’ aggregated hunches about election results. Results from four major German elections show that mere recognition of party names forecast the parties’ electoral success fairly well. Recognition-based forecasts were most competitive with the other models when forecasting the smaller parties’ success and for small sample sizes. However, wisdom-of-crowds-forecasts outperformed recognition-based forecasts in most cases. It seems that wisdom-of-crowds-forecasts are able to draw on the benefits of recognition while at the same time avoiding its downsides, such as lack of discrimination among very famous parties or recognition caused by factors unrelated to electoral success. Yet it seems that a simple extension of the recognition-based forecasts—asking people what proportion of the population would recognize a party instead of whether they themselves recognize it—is also able to eliminate these downsides.

Four challenges for cognitive research on the recognition heuristic and a call for a research strategy shift

The recognition heuristic assumes that people make inferences based on the output of recognition memory. While much work has been devoted to establishing the recognition heuristic as a viable description of how people make inferences, more work is needed to fully integrate research on the recognition heuristic with research from the broader cognitive psychology literature. In this article, we outline four challenges that should be met for this integration to take place, and close with a call to address these four challenges collectively, rather than piecemeal.

Decision-making under risk conditions is susceptible to interference by a secondary executive task

Recent research suggests two ways of making decisions: an intuitive and an analytical one. The current study examines whether a secondary executive task interferes with advantageous decision-making in the Game of Dice Task (GDT), a decision-making task with explicit and stable rules that taps executive functioning. One group of participants performed the original GDT solely, two groups performed either the GDT and a 1-back or a 2-back working memory task as a secondary task simultaneously. Results show that the group which performed the GDT and the secondary task with high executive load (2-back) decided less advantageously than the group which did not perform a secondary executive task. These findings give further evidence for the view that decision-making under risky conditions taps into the rational-analytical system which acts in a serial and not parallel way as performance on the GDT is disturbed by a parallel task that also requires executive resources.