Another chance for good reasoning

The impact of problem domain on Bayesian inferences: A systematic investigation

Sparse (and occasionally contradictory) evidence exists regarding the impact of domain on probabilistic updating, some of which suggests that Bayesian word problems with medical content may be especially challenging. The present research aims to address this gap in knowledge through three pre-registered online studies, which involved a total of 2,238 participants. Bayesian word problems were related to one of three domains: medical, daily-life, and abstract. In the first two cases, problems presented realistic content and plausible numerical information, while in the latter, problems contained explicitly imaginary elements. Problems across domains were matched in terms of all relevant statistical values and, as much as possible, wording. Studies 1 and 2 utilized the same set of problems, but different response elicitation methods (i.e., an open-ended and a multiple-choice question, respectively). Study 3 involved a larger number of participants per condition and a smaller set of problems to more thoroughly investigate the magnitude of differences between the domains. There was a generally low rate of correct responses (17.2%, 17.4%, and 14.3% in Studies 1, 2, and 3, respectively), consistent with accuracy levels commonly observed in the literature for this specific task with online samples. Nonetheless, a small but significant difference between domains was observed: participants' accuracy did not differ between medical and daily-life problems, while it was significantly higher in corresponding abstract problems. These results suggest that medical problems are not inherently more difficult to solve, but rather that performance is improved with abstract problems for which participants cannot draw from their background knowledge.

When intuitive Bayesians need to be good readers: The problem-wording effect on Bayesian reasoning

Are humans intuitive Bayesians? It depends. People seem to be Bayesians when updating probabilities from experience but not when acquiring probabilities from descriptions (i.e., Bayesian textbook problems). Decades of research on textbook problems have focused on how the format of the statistical information (e.g., the natural frequency effect) affects such reasoning. However, it pays much less attention to the wording of these problems. Mathematical problem-solving literature indicates that wording is critical for performance. Wording effects (the wording varied across the problems and manipulations) can also have far-reaching consequences. These may have confounded between-format comparisons and moderated within-format variability in prior research. Therefore, across seven experiments (N = 4909), we investigated the impact of the wording of medical screening problems and statistical formats on Bayesian reasoning in a general adult population. Participants generated more Bayesian answers with natural frequencies than with single-event probabilities, but only with the improved wording. The improved wording of the natural frequencies consistently led to more Bayesian answers than the natural frequencies with standard wording. The improved wording effect occurred mainly due to a more efficient description of the statistical information-cueing required mathematical operations, an unambiguous association of numbers with their reference class and verbal simplification. The wording effect extends the current theoretical explanations of Bayesian reasoning and bears methodological and practical implications. Ultimately, even intuitive Bayesians must be good readers when solving Bayesian textbook problems.

What facilitates Bayesian reasoning? A crucial test of ecological rationality versus nested sets hypotheses

Different theoretical views about Bayesian reasoning (ecological rationality and nested sets views) both claim support from results showing that natural sampling, whole numbers, and pictorial representations help with reasoning performance, although they differ in explaining how those results occur. Three studies (total N = 653) use minimally different numerical presentation formats-varying the singular or plural tense of the context story topic-and presence or absence of an additional icon array picture, to better understand the mechanisms driving these reasoning performance results. Plural wording, indicating a conceptual aggregation (i.e., frequencies) rather than just numerical whole numbers, consistently boosted performance. Icon arrays, in contrast, were helpful only when alongside single-tense information. These results fit more consistently with an ecological rationality view which has long argued that the mind is adapted to work best with frequentist information.

Processing Probability Information in Nonnumerical Settings - Teachers' Bayesian and Non-bayesian Strategies During Diagnostic Judgment

A diagnostic judgment of a teacher can be seen as an inference from manifest observable evidence on a student’s behavior to his or her latent traits. This can be described by a Bayesian model of inference: The teacher starts from a set of assumptions on the student (hypotheses), with subjective probabilities for each hypothesis (priors). Subsequently, he or she uses observed evidence (students’ responses to tasks) and knowledge on conditional probabilities of this evidence (likelihoods) to revise these assumptions. Many systematic deviations from this model (biases, e.g., base-rate neglect, inverse fallacy) are reported in the literature on Bayesian reasoning. In a teacher’s situation, the information (hypotheses, priors, likelihoods) is usually not explicitly represented numerically (as in most research on Bayesian reasoning) but only by qualitative estimations in the mind of the teacher. In our study, we ask to which extent individuals (approximately) apply a rational Bayesian strategy or resort to other biased strategies of processing information for their diagnostic judgments. We explicitly pose this question with respect to nonnumerical settings. To investigate this question, we developed a scenario that visually displays all relevant information (hypotheses, priors, likelihoods) in a graphically displayed hypothesis space (called “hypothegon”)–without recurring to numerical representations or mathematical procedures. Forty-two preservice teachers were asked to judge the plausibility of different misconceptions of six students based on their responses to decimal comparison tasks (e.g., 3.39 > 3.4). Applying a Bayesian classification procedure, we identified three updating strategies: a Bayesian update strategy (BUS, processing all probabilities), a combined evidence strategy (CES, ignoring the prior probabilities but including all likelihoods), and a single evidence strategy (SES, only using the likelihood of the most probable hypothesis). In study 1, an instruction on the relevance of using all probabilities (priors and likelihoods) only weakly increased the processing of more information. In study 2, we found strong evidence that a visual explication of the prior–likelihood interaction led to an increase in processing the interaction of all relevant information. These results show that the phenomena found in general research on Bayesian reasoning in numerical settings extend to diagnostic judgments in nonnumerical settings.

Fostering the Understanding of Positive Test Results

Background

The majority of health-service users seem unable to properly compute the positive predictive value of medical tests. The research reported in the present study sought to investigate whether, and to what extent, probabilistic inferences about a positive test result can be improved by changing the traditional way in which probability judgments are elicited and medical information is presented.

Methods

Online survey respondents were presented with a positive test result regarding a pregnant woman, and had to estimate the chances that her unborn baby had an anomaly (standard judgment), to apportion the numbers of chances for and against this hypothesis (distributive judgment), and to indicate whether the hypothesis that the baby had an anomaly was more or less likely than its alternative (relative judgment). Test sensitivity and information framing were also manipulated.

Results

Irrespective of education and to some extent of numeracy, the majority of respondents produced correct distributive assessments of chances, which were in line with relative judgments and more accurate than standard ones. When information displayed exclusively positive test results, inferences resulted further improved and unaffected by test sensitivity.

Conclusions

Simple communication strategies that prompt extensional reasoning on the relevant set of number of chances can help individuals to overcome probabilistic inference errors.

From reading numbers to seeing ratios: a benefit of icons for risk comprehension

Promoting a better understanding of statistical data is becoming increasingly important for improving risk comprehension and decision-making. In this regard, previous studies on Bayesian problem solving have shown that iconic representations help infer frequencies in sets and subsets. Nevertheless, the mechanisms by which icons enhance performance remain unclear. Here, we tested the hypothesis that the benefit offered by icon arrays lies in a better alignment between presented and requested relationships, which should facilitate the comprehension of the requested ratio beyond the represented quantities. To this end, we analyzed individual risk estimates based on data presented either in standard verbal presentations (percentages and natural frequency formats) or as icon arrays. Compared to the other formats, icons led to estimates that were more accurate, and importantly, promoted the use of equivalent expressions for the requested probability. Furthermore, whereas the accuracy of the estimates based on verbal formats depended on their alignment with the text, all the estimates based on icons were equally accurate. Therefore, these results support the proposal that icons enhance the comprehension of the ratio and its mapping onto the requested probability and point to relational misalignment as potential interference for text-based Bayesian reasoning. The present findings also argue against an intrinsic difficulty with understanding single-event probabilities.