Ecological and evolutionary validity: comments on Johnson-Laird, Legrenzi, Girotto, Legrenzi, and Caverni's (1999) mental-model theory of extensional reasoning

What facilitates Bayesian reasoning? A crucial test of ecological rationality versus nested sets hypotheses

Different theoretical views about Bayesian reasoning (ecological rationality and nested sets views) both claim support from results showing that natural sampling, whole numbers, and pictorial representations help with reasoning performance, although they differ in explaining how those results occur. Three studies (total N = 653) use minimally different numerical presentation formats-varying the singular or plural tense of the context story topic-and presence or absence of an additional icon array picture, to better understand the mechanisms driving these reasoning performance results. Plural wording, indicating a conceptual aggregation (i.e., frequencies) rather than just numerical whole numbers, consistently boosted performance. Icon arrays, in contrast, were helpful only when alongside single-tense information. These results fit more consistently with an ecological rationality view which has long argued that the mind is adapted to work best with frequentist information.

Participant recruitment methods and statistical reasoning performance

Optimal Bayesian reasoning performance has reportedly been elusive, and a variety of explanations have been suggested for this situation. In a series of experiments, it is demonstrated that these difficulties with replication can be accounted for by differences in participant-sampling methodologies. Specifically, the best performances are obtained with students from top-tier, national universities who were paid for their participation. Performance drops significantly as these conditions are altered regarding inducements (e.g., using unpaid participants) or participant source (e.g., using participants from a second-tier, regional university). Honours-programme undergraduates do better than regular undergraduates within the same university, paid participation creates superior performance, and top-tier university students do better than students from lower ranked universities. Pictorial representations (supplementing problem text) usually have a slight facilitative effect across these participant manipulations. These results indicate that studies should take account of these methodological details and focus more on relative levels of performance rather than absolute performance.

Another chance for good reasoning

Disagreement on the "probability status" of chances casts doubt on Girotto and Gonzalez's (2001) conclusion that the human mind can make sound Bayesian inferences involving single-event probabilities. The main objection raised has been that chances are de facto natural frequencies disguised as probabilities. In the present study, we empirically demonstrated that numbers of chances are perceived as being distinct from natural frequencies and that they have a facilitatory effect on Bayesian inference tasks that is completely independent from their (minor) frequentist readings. Overall, therefore, our results strongly disconfirm the hypothesis that natural frequencies are a privileged cognitive representational format for Bayesian inferences and suggest that a significant portion of laypeople adequately handle genuine single-event probability problems once these are rendered computationally more accessible by using numbers of chances.

Helping Doctors and Patients Make Sense of Health Statistics

Many doctors, patients, journalists, and politicians alike do not understand what health statistics mean or draw wrong conclusions without noticing. Collective statistical illiteracy refers to the widespread inability to understand the meaning of numbers. For instance, many citizens are unaware that higher survival rates with cancer screening do not imply longer life, or that the statement that mammography screening reduces the risk of dying from breast cancer by 25% in fact means that 1 less woman out of 1,000 will die of the disease. We provide evidence that statistical illiteracy (a) is common to patients, journalists, and physicians; (b) is created by nontransparent framing of information that is sometimes an unintentional result of lack of understanding but can also be a result of intentional efforts to manipulate or persuade people; and (c) can have serious consequences for health.

The causes of statistical illiteracy should not be attributed to cognitive biases alone, but to the emotional nature of the doctor–patient relationship and conflicts of interest in the healthcare system. The classic doctor–patient relation is based on (the physician's) paternalism and (the patient's) trust in authority, which make statistical literacy seem unnecessary; so does the traditional combination of determinism (physicians who seek causes, not chances) and the illusion of certainty (patients who seek certainty when there is none). We show that information pamphlets, Web sites, leaflets distributed to doctors by the pharmaceutical industry, and even medical journals often report evidence in nontransparent forms that suggest big benefits of featured interventions and small harms. Without understanding the numbers involved, the public is susceptible to political and commercial manipulation of their anxieties and hopes, which undermines the goals of informed consent and shared decision making.

What can be done? We discuss the importance of teaching statistical thinking and transparent representations in primary and secondary education as well as in medical school. Yet this requires familiarizing children early on with the concept of probability and teaching statistical literacy as the art of solving real-world problems rather than applying formulas to toy problems about coins and dice. A major precondition for statistical literacy is transparent risk communication. We recommend using frequency statements instead of single-event probabilities, absolute risks instead of relative risks, mortality rates instead of survival rates, and natural frequencies instead of conditional probabilities. Psychological research on transparent visual and numerical forms of risk communication, as well as training of physicians in their use, is called for.

Statistical literacy is a necessary precondition for an educated citizenship in a technological democracy. Understanding risks and asking critical questions can also shape the emotional climate in a society so that hopes and anxieties are no longer as easily manipulated from outside and citizens can develop a better-informed and more relaxed attitude toward their health.

Good fences make for good neighbors but bad science: a review of what improves Bayesian reasoning and why

Bayesian reasoning, defined here as the updating of a posterior probability following new information, has historically been problematic for humans. Classic psychology experiments have tested human Bayesian reasoning through the use of word problems and have evaluated each participant’s performance against the normatively correct answer provided by Bayes’ theorem. The standard finding is of generally poor performance. Over the past two decades, though, progress has been made on how to improve Bayesian reasoning. Most notably, research has demonstrated that the use of frequencies in a natural sampling framework—as opposed to single-event probabilities—can improve participants’ Bayesian estimates. Furthermore, pictorial aids and certain individual difference factors also can play significant roles in Bayesian reasoning success. The mechanics of how to build tasks which show these improvements is not under much debate. The explanations for why naturally sampled frequencies and pictures help Bayesian reasoning remain hotly contested, however, with many researchers falling into ingrained “camps” organized around two dominant theoretical perspectives. The present paper evaluates the merits of these theoretical perspectives, including the weight of empirical evidence, theoretical coherence, and predictive power. By these criteria, the ecological rationality approach is clearly better than the heuristics and biases view. Progress in the study of Bayesian reasoning will depend on continued research that honestly, vigorously, and consistently engages across these different theoretical accounts rather than staying “siloed” within one particular perspective. The process of science requires an understanding of competing points of view, with the ultimate goal being integration.

Using statistical reasoning performance to reveal information parsing preferences in the mind

Many cognitive tasks require the parsing of information into smaller, discrete units in order to enable effective information processing. This parsing can, broadly speaking, be done along either situationally ad hoc dimensions or done preferentially along ecologically and evolutionarily relevant dimensions. The present research systematically evaluates these two possibilities within a statistical reasoning context. While replicating results that appear to support the partition-edit-count hypothesis (that item parsing is equipotential, based on subtle linguistic cues), this result was found to be in large part due to confounds in the nature of the tasks rather than the partitioning manipulations (Experiment 1). Additionally, a frequency presentation of the same task not only eliminated the earlier confounds but also improved performance directly and as predicted by the alternative hypothesis (Experiment 2). Attempts to reintroduce a biasing partition frame (Experiment 3) and a process study of participants' task representation (Experiment 4) also both failed to support the partition-edit-count hypothesis. These results favour an ecological rationality perspective and the associated frequency and individuation hypotheses regarding statistical reasoning (i.e., a privileged status for frequency representations to guide parsing of objects, events, and locations into easily countable units).

When and for whom do frequencies facilitate performance? On the role of numerical literacy

The thesis that the mind is better prepared to process frequencies-as compared to other numerical formats-continues to be debated. A recent aspect of this issue is the role of numeracy (numerical literacy; one's ability to understand and work with numerical information) and specifically the argument that individual differences in numeracy interact with numerical formats. This interaction, either that frequencies improve performance only for those of low numeracy or that frequencies work only for those of high numeracy, would suggest that better performance using frequencies could be due to (nonevolutionary) numeracy effects. The three present studies revisited prior work with cumulative probability, Bayesian reasoning, and scenario risk assessments to study the effects of numeracy on frequency facilitation. Results from these experiments consistently failed to replicate previous findings of interactions; however, a more consistent finding emerged of a straightforward frequency effect. The lack of interactions and observations of frequency main effects lend support to the evolutionary explanation of the frequency effect. In addition, some possible statistical processes are proposed to explain the observation of interactions in past studies.

How numeracy influences risk comprehension and medical decision making

We review the growing literature on health numeracy, the ability to understand and use numerical information, and its relation to cognition, health behaviors, and medical outcomes. Despite the surfeit of health information from commercial and noncommercial sources, national and international surveys show that many people lack basic numerical skills that are essential to maintain their health and make informed medical decisions. Low numeracy distorts perceptions of risks and benefits of screening, reduces medication compliance, impedes access to treatments, impairs risk communication (limiting prevention efforts among the most vulnerable), and, based on the scant research conducted on outcomes, appears to adversely affect medical outcomes. Low numeracy is also associated with greater susceptibility to extraneous factors (i.e., factors that do not change the objective numerical information). That is, low numeracy increases susceptibility to effects of mood or how information is presented (e.g., as frequencies vs. percentages) and to biases in judgment and decision making (e.g., framing and ratio bias effects). Much of this research is not grounded in empirically supported theories of numeracy or mathematical cognition, which are crucial for designing evidence-based policies and interventions that are effective in reducing risk and improving medical decision making. To address this gap, we outline four theoretical approaches (psychophysical, computational, standard dual-process, and fuzzy trace theory), review their implications for numeracy, and point to avenues for future research.

How different types of participant payments alter task performance

Researchers typically use incentives (such as money or course credit) in order to obtain participants who engage in the specific behaviors of interest to the researcher. There is, however, little understanding or agreement on the effects of different types and levels of incentives used. Some results in the domain of statistical reasoning suggest that performance differences — previously deemed theoretically important — may actually be due to differences in incentive types across studies. 704 participants completed one of five variants of a statistical reasoning task, for which they received either course credit, flat fee payment, or performance-based payment incentives. Successful task completion was more frequent with performance-based incentives than with either of the other incentive types. Performance on moderately difficult tasks (compared to very easy and very hard tasks) was most sensitive to incentives. These results can help resolve existing debates about inconsistent findings, guide more accurate comparisons across studies, and be applied beyond research settings.

Base-rate respect: From ecological rationality to dual processes

The phenomenon of base-rate neglect has elicited much debate. One arena of debate concerns how people make judgments under conditions of uncertainty. Another more controversial arena concerns human rationality. In this target article, we attempt to unpack the perspectives in the literature on both kinds of issues and evaluate their ability to explain existing data and their conceptual coherence. From this evaluation we conclude that the best account of the data should be framed in terms of a dual-process model of judgment, which attributes base-rate neglect to associative judgment strategies that fail to adequately represent the set structure of the problem. Base-rate neglect is reduced when problems are presented in a format that affords accurate representation in terms of nested sets of individuals.

Omissions, conflations, and false dichotomies: Conceptual and empirical problems with the Barbey & Sloman account

Both the theoretical frameworks that organize the first part of Barbey & Sloman's (B&S's) target article and the empirical evidence marshaled in the second part are marked by distinctions that should not exist (i.e., false dichotomies), conflations where distinctions should be made, and selective omissions of empirical results – within the very studies discussed – that create illusions of theoretical and empirical favor.

Children can solve Bayesian problems: the role of representation in mental computation

Can children reason the Bayesian way? We argue that the answer to this question depends on how numbers are represented, because a representation can do part of the computation. We test, for the first time, whether Bayesian reasoning can be elicited in children by means of natural frequencies. We show that when information was presented to fourth, fifth, and sixth graders in terms of probabilities, their ability to estimate the Bayesian posterior probability was zero. Yet when the same information was presented in natural frequencies, Bayesian reasoning showed a steady increase from fourth to sixth grade, reaching an average level of 19, 39, and 53%, respectively, in two studies. Sixth graders' performance with natural frequencies matched the performance of adults with probabilities. But this general increase was accompanied by striking individual differences. More than half of the sixth graders solved most or all problems, whereas one third could not solve a single one. An analysis of the children's responses provides evidence for the use of three non-Bayesian strategies. These follow an overlapping wave model of development and continue to be observed in the minds of adults. More so than adults' probabilistic reasoning, children's reasoning depends on a proper representation of information.