Visual representation of statistical information improves diagnostic inferences in doctors and their patients

[The expert panel for Script Concordance Tests: A truly adequate reference?]

The Script Concordance Tests (SCTs) are an examination modality introduced by decree in the French National Ranking Exam for medical students in 2024. Their objective is to evaluate clinical reasoning in situations of uncertainty. In practice, SCTs assess the impact of new information on the probability of a hypothesis formulated a priori based on an authentic clinical scenario. This approach resembles probabilistic (or Bayesian) reasoning. Due to the uncertainty associated with the explored clinical situation, SCTs do not compare the student's response to an expected one in a theoretical knowledge reference. Instead, the distribution of responses from a panel of experienced physicians is used to establish the question's scoring scale. Literature data suggest that physicians, even experienced ones, like most humans, often exhibit biased intuitive probabilistic reasoning. These biases raise questions about the relevance of using expert panel responses as scoring scales for SCTs.

Numerical versus graphical aids for decision-making in a multi-cue signal identification task

Decision aids are commonly used in tactical decision-making environments to help humans integrate base-rate and multi-cue information. However, it is important that users appropriately trust and rely on aids. Decision aids can be presented in many ways, but the literature lacks clarity over the conditions surrounding their effectiveness. This research aims to determine whether a numerical or graphical aid more effectively supports human performance, and explores the relationships between aid presentation, trust, and workload. Participants (N = 30) completed a signal-identification task that required integration of readings from a set of three dynamic gauges. Participants experienced three conditions: unaided, using a numerical aid, and using a graphical aid. The aids combined gauge and base-rate information in a statistically-optimal fashion. Participants also indicated how much they trusted the system and how hard they worked during the task. Analyses explored the impact of aid condition on sensitivity, response bias, response time, trust, and workload. Both the numerical and graphical aids produced significant increases in sensitivity and trust, and significant decreases in workload in comparison to the unaided condition. The difference in response time between the graphical and unaided conditions approached significance, with participants responding faster using the graphical aid without decrements in sensitivity. Significant interactions between aid and signal type indicated that both aided conditions promoted faster responding to non-hostile signals, with larger mean differences in the graphical aid condition. Practically, graphical aids in which suggestions are more salient to users may promote faster responding in tactical environments, with negligible cost of accuracy.

Patient decision-making in the era of transcarotid artery revascularization

OBJECTIVE

The National Coverage Determination on carotid stenting by Medicare in October 2023 stipulates that patients participate in a shared decision-making (SDM) conversation with their proceduralist before an intervention. However, to date, there is no validated SDM tool that incorporates transcarotid artery revascularization (TCAR) into its decision platform. Our objective was to elicit patient and surgeon experiences and preferences through a qualitative approach to better inform the SDM process surrounding carotid revascularization.

METHODS

We performed longitudinal perioperative semistructured interviews of 20 participants using purposive maximum variation sampling, a qualitative technique designed for identification and selection of information-rich cases, to define domains important to participants undergoing carotid endarterectomy or TCAR and impressions of SDM. We also performed interviews with nine vascular surgeons to elicit their input on the SDM process surrounding carotid revascularization. Interview data were coded and analyzed using inductive content analysis coding.

RESULTS

We identified three important domains that contribute to the participants' ultimate decision on which procedure to choose: their individual values, their understanding of the disease and each procedure, and how they prefer to make medical decisions. Participant values included themes such as success rates, "wanting to feel better," and the proceduralist's experience. Participants varied in their desired degree of understanding of carotid disease, but all individuals wished to discuss each option with their proceduralist. Participants' desired medical decision-making style varied on a spectrum from complete autonomy to wanting the proceduralist to make the decision for them. Participants who preferred carotid endarterectomy felt outcomes were superior to TCAR and often expressed a desire to eliminate the carotid plaque. Those selecting TCAR felt it was a newer, less invasive option with the shortest procedural and recovery times. Surgeons frequently noted patient factors such as age and anatomy, as well as the availability of long-term data, as reasons to preferentially select one procedure. For most participants, their surgeon was viewed as the most important source of information surrounding their disease and procedure.

CONCLUSIONS

SDM surrounding carotid revascularization is nuanced and marked by variation in patient preferences surrounding autonomy when choosing treatment. Given the mandate by Medicare to participate in a SDM interaction before carotid stenting, this analysis offers critical insights that can help to guide an efficient and effective dialog between patients and providers to arrive at a shared decision surrounding therapeutic intervention for patients with carotid disease.

Disclosing Results of Tests for Covert Consciousness: A Framework for Ethical Translation

The advent of neurotechnologies including advanced functional magnetic resonance imaging and electroencephalography to detect states of awareness not detectable by traditional bedside neurobehavioral techniques (i.e., covert consciousness) promises to transform neuroscience research and clinical practice for patients with brain injury. As these interventions progress from research tools into actionable, guideline-endorsed clinical tests, ethical guidance for clinicians on how to responsibly communicate the sensitive results they yield is crucial yet remains underdeveloped. Drawing on insights from empirical and theoretical neuroethics research and our clinical experience with advanced neurotechnologies to detect consciousness in behaviorally unresponsive patients, we critically evaluate ethical promises and perils associated with disclosing the results of clinical covert consciousness assessments and describe a semistructured approach to responsible data sharing to mitigate potential risks.

The impact of problem domain on Bayesian inferences: A systematic investigation

Sparse (and occasionally contradictory) evidence exists regarding the impact of domain on probabilistic updating, some of which suggests that Bayesian word problems with medical content may be especially challenging. The present research aims to address this gap in knowledge through three pre-registered online studies, which involved a total of 2,238 participants. Bayesian word problems were related to one of three domains: medical, daily-life, and abstract. In the first two cases, problems presented realistic content and plausible numerical information, while in the latter, problems contained explicitly imaginary elements. Problems across domains were matched in terms of all relevant statistical values and, as much as possible, wording. Studies 1 and 2 utilized the same set of problems, but different response elicitation methods (i.e., an open-ended and a multiple-choice question, respectively). Study 3 involved a larger number of participants per condition and a smaller set of problems to more thoroughly investigate the magnitude of differences between the domains. There was a generally low rate of correct responses (17.2%, 17.4%, and 14.3% in Studies 1, 2, and 3, respectively), consistent with accuracy levels commonly observed in the literature for this specific task with online samples. Nonetheless, a small but significant difference between domains was observed: participants' accuracy did not differ between medical and daily-life problems, while it was significantly higher in corresponding abstract problems. These results suggest that medical problems are not inherently more difficult to solve, but rather that performance is improved with abstract problems for which participants cannot draw from their background knowledge.

Game-based learning to improve diagnostic accuracy: a pilot randomized-controlled trial

OBJECTIVES

Perform a pilot study of online game-based learning (GBL) using natural frequencies and feedback to teach diagnostic reasoning.

METHODS

We conducted a multicenter randomized-controlled trial of computer-based training. We enrolled medical students, residents, practicing physicians and nurse practitioners. The intervention was a 45 min online GBL training vs. control education with a primary outcome of score on a scale of diagnostic accuracy (composed of 10 realistic case vignettes, requesting estimates of probability of disease after a test result, 0-100 points total).

RESULTS

Of 90 participants there were 30 students, 30 residents and 30 practicing clinicians. Of these 62 % (56/90) were female and 52 % (47/90) were white. Sixty were randomized to GBL intervention and 30 to control. The primary outcome of diagnostic accuracy immediately after training was better in GBL (mean accuracy score 59.4) vs. control (37.6), p=0.0005. The GBL group was then split evenly (30, 30) into no further intervention or weekly emails with case studies. Both GBL groups performed better than control at one-month and some continued effect at three-month follow up. Scores at one-month GBL (59.2) GBL plus emails (54.2) vs. control (33.9), p=0.024; three-months GBL (56.2), GBL plus emails (42.9) vs. control (35.1), p=0.076. Most participants would recommend GBL to colleagues (73 %), believed it was enjoyable (92 %) and believed it improves test interpretation (95 %).

CONCLUSIONS

In this pilot study, a single session with GBL nearly doubled score on a scale of diagnostic accuracy in medical trainees and practicing clinicians. The impact of GBL persisted after three months.

How general is the natural frequency effect? The case of joint probabilities

Natural frequencies are known to improve performance in Bayesian reasoning. However, their impact in situations with two binary events has not yet been completely examined, as most researchers in the last 30 years focused only on conditional probabilities. Nevertheless, situations with two binary events consist of 16 elementary probabilities and so we widen the scope and focus on joint probabilities. In this article, we theoretically elaborate on the importance of joint probabilities, for example, in situations like the Linda problem. Furthermore, we implemented a study in a 2×5×2 design with the factors information format (probabilities vs. natural frequencies), visualization type ("Bayesian text" vs. tree diagram vs. double tree diagram vs. net diagram vs. 2×2 table), and context (mammography vs. economics problem). Additionally, all four "joint questions" (i.e., P ( A ∩ B ) , P ( A ¯ ∩ B ) , P ( A ¯ ∩ B ¯ ) , P ( A ∩ B ¯ ) ) were asked for. The main factor of interest was whether there is a format effect in the five visualization types named above. Surprisingly, the advantage of natural frequencies was not found for joint probabilities and, most strikingly, the format interacted with the visualization type. Specifically, while people's understanding of joint probabilities in a double tree seems to be worse than the understanding of the corresponding natural frequencies (and, thus, the frequency effect holds true), the opposite seems to be true in the 2 × 2 table. Hence, the advantage of natural frequencies compared to probabilities in typical Bayesian tasks cannot be found in the same way when joint probability or frequency tasks are asked.

When intuitive Bayesians need to be good readers: The problem-wording effect on Bayesian reasoning

Are humans intuitive Bayesians? It depends. People seem to be Bayesians when updating probabilities from experience but not when acquiring probabilities from descriptions (i.e., Bayesian textbook problems). Decades of research on textbook problems have focused on how the format of the statistical information (e.g., the natural frequency effect) affects such reasoning. However, it pays much less attention to the wording of these problems. Mathematical problem-solving literature indicates that wording is critical for performance. Wording effects (the wording varied across the problems and manipulations) can also have far-reaching consequences. These may have confounded between-format comparisons and moderated within-format variability in prior research. Therefore, across seven experiments (N = 4909), we investigated the impact of the wording of medical screening problems and statistical formats on Bayesian reasoning in a general adult population. Participants generated more Bayesian answers with natural frequencies than with single-event probabilities, but only with the improved wording. The improved wording of the natural frequencies consistently led to more Bayesian answers than the natural frequencies with standard wording. The improved wording effect occurred mainly due to a more efficient description of the statistical information-cueing required mathematical operations, an unambiguous association of numbers with their reference class and verbal simplification. The wording effect extends the current theoretical explanations of Bayesian reasoning and bears methodological and practical implications. Ultimately, even intuitive Bayesians must be good readers when solving Bayesian textbook problems.

Decision aids for patients with carotid stenosis

BACKGROUND

Shared decision-making tools have been underused by clinicians in real-world practice. Changes to the National Coverage Determination by Medicare for carotid stenting greatly expand the coverage for patients, but simultaneously require a shared decision-making interaction that involves the use of a validated tool. Accordingly, our objective was to evaluate the currently available decision aids for carotid stenosis.

METHODS

We conducted a review of the literature for published work on decision aids for the treatment of carotid disease.

RESULTS

Four publications met inclusion criteria. We found the format of the decision aid impacted patient comprehension and decision making, although patient characteristics also played a role in the therapeutic decisions made. Notably, none of the available decision aids included the widely adopted transcarotid artery revascularization as an option.

CONCLUSIONS

Further work is needed in the development of a widespread validated decision aid instrument for patients with carotid stenosis.

Previous beliefs affect Bayesian reasoning in conditions fostering gist comprehension

It has been shown that Bayesian reasoning is affected by the believability of the data, but it is unknown which conditions could potentiate or reduce such belief effect. Here, we tested the hypothesis that the belief effect would mainly be observed in conditions fostering a gist comprehension of the data. Accordingly, we expected to observe a significant belief effect in iconic rather than in textual presentations and, in general, when nonnumerical estimates were requested. The results of three studies showed more accurate Bayesian estimates, either expressed numerically or nonnumerically, for icons than for text descriptions of natural frequencies. Moreover, in line with our expectations, nonnumerical estimates were, in general, more accurate for believable rather than for unbelievable scenarios. In contrast, the belief effect on the accuracy of the numerical estimates depended on the format and on the complexity of the calculation. The present findings also showed that single-event posterior probability estimates based on described frequencies were more accurate when expressed nonnumerically rather than numerically, opening new avenues for the development of interventions to improve Bayesian reasoning.

Measuring people's covariational reasoning in Bayesian situations

Previous research on Bayesian reasoning has typically investigated people's ability to assess a posterior probability (i.e., a positive predictive value) based on prior knowledge (i.e., base rate, true-positive rate, and false-positive rate). In this article, we systematically examine the extent to which people understand the effects of changes in the three input probabilities on the positive predictive value, that is, covariational reasoning. In this regard, two different operationalizations for measuring covariational reasoning (i.e., by single-choice vs. slider format) are investigated in an empirical study with N = 229 university students. In addition, we aim to answer the question wheter a skill in "conventional" Bayesian reasoning is a prerequisite for covariational reasoning.

The Use of Visualizations to Improve Bayesian Reasoning: A Literature Review

Decisions are often made under uncertainty. The most that one can do is use prior knowledge (e.g., base rates, prior probabilities, etc.) and make the most probable choice given the information we have. Unfortunately, most people struggle with Bayesian reasoning. Poor performance within Bayesian reasoning problems has led researchers to investigate ways to improve Bayesian reasoning. Many have found success in using natural frequencies instead of probabilities to frame problems. Beyond the quantitative format, there is growing literature on the use of visualizations or visual representations to improve Bayesian reasoning, which will be the focus of this review. In this review, we discuss studies that have found visualizations to be effective for improving Bayesian reasoning in a lab or classroom setting and discuss the considerations for using visualizations, paying special attention to individual differences. In addition, we will review the factors that influence Bayesian reasoning, such as natural frequencies vs. probabilities, problem format, individual differences, and interactivity. We also provide general and specific suggestions for future research.

Boosting for insight and/or boosting for agency? How to maximize accurate test interpretation with natural frequencies

BACKGROUND

Many physicians do not know how to accurately interpret test results using Bayes' rule. As a remedy, two kinds of interventions have been shown effective: boosting insight and boosting agency with natural frequencies. To boost insight, test statistics are provided in natural frequencies (rather than conditional probabilities), without instructions on how to use them. To boost agency, a training is provided on how to translate probabilities into natural frequencies and apply them in Bayes' rule. What has not been shown is whether boosting agency is sufficient or if representing test statistics in natural frequencies may additionally boost insight to maximize accurate test interpretation.

METHODS

We used a pre/posttest design to assess test interpretation accuracy of 577 medical students before and after a training on two Bayesian reasoning tasks, one providing conditional probabilities, the other natural frequencies. The pretest assessed baseline abilities versus the effect of natural frequencies to boost insight. After participants received a training on how to translate conditional probabilities into natural frequencies and how to apply them in Bayes' rule, test interpretation skills were assessed using the same tasks again, comparing the effects of training-induced agency with versus without additionally boosting insight (i.e., test statistics in natural frequencies versus conditional probabilities).

RESULTS

Compared to the test question formatted in conditional probabilities (34% correct answers), natural frequencies facilitated Bayesian reasoning without training (68%), that is, they increased insight. The training on how to use natural frequencies improved performance for tasks formatted in conditional probabilities (64%). Performance was maximal after training and with test statistics formatted in natural frequencies, that is, with a combination of boosting insight and agency (89%).

CONCLUSIONS

Natural frequencies should be used to boost insight and agency to maximize effective use of teaching resources. Thus, mandating that test statistics are provided in natural frequencies and adopting short trainings on how to translate conditional probabilities into natural frequencies and how to apply them in Bayes' rule will help to maximize accurate test interpretation.

TRIAL REGISTRATION

The study was a registered with the German Clinical Trial Registry ( DRKS00008723 ; 06/03/2015).

Investigating the presentation of uncertainty in an icon array: A randomized trial

BACKGROUND

Clinicians are often advised to use pictographs to communicate risk, but whether they offer benefits when communicating risk imprecision (e.g., 65%-79%) is unknown.

PURPOSE

To test whether any of three approaches to visualizing imprecision would more effectively communicate breast and ovarian cancer risk for BRCA1 pathogenic variant carriers.

METHODS

1,300 UK residents were presented with a genetic report with information about BRCA1-related risks, with random assignment to one of four formats: no visualization (text alone), or a pictograph using shaded icons, a gradient, or arrows marking range endpoints. We also tested pictographs in two layouts. Analysis of variance (ANOVA) and regression was employed.

RESULTS

There was no effect of format. Participants shown pictographs vs. text alone had better uptake of breast cancer risk messages (p < .05, η ² = 0.003). Pictographs facilitated memory for the specific amount of risk (p < 0.001, η ² = 0.019), as did the tabular layout. Individuals not having completed upper secondary education may benefit most.

CONCLUSIONS

We found weak evidence in favor of using simple pictographs with ranges to communicate BRCA risk (versus text alone), and of the tabular layout.

INNOVATION

Testing different ways of communicating imprecision within pictographs is a novel and promising line of research.

Reference Dependence in Bayesian Reasoning: Value Selection Bias, Congruence Effects, and Response Prompt Sensitivity

This work examines the influence of reference dependence, including value selection bias and congruence effects, on diagnostic reasoning. Across two studies, we explored how dependence on the initial problem structure influences the ability to solve simplified precursors to the more traditional Bayesian reasoning problems. Analyses evaluated accuracy and types of response errors as a function of congruence between the problem presentation and question of interest, amount of information, need for computation, and individual differences in numerical abilities. Across all problem variations, there was consistent and strong evidence of a value selection bias in that incorrect responses almost always conformed to values that were provided in the problem rather than other errors including those related to computation. The most consistent and unexpected error across all conditions in the first experiment was that people were often more likely to utilize the superordinate value (N) as part of their solution rather than the anticipated reference class values. This resulted in a weakened effect of congruence, with relatively low accuracy even in congruent conditions, and a dominant response error of the superordinate value. Experiment 2 confirmed that the introduction of a new sample drew attention away from the provided reference class, increasing reliance on the overall sample size. This superordinate preference error, along with the benefit of repeating the PPV reference class within the question, demonstrated the importance of reference dependence based on the salience of information within the response prompt. Throughout, higher numerical skills were generally associated with higher accuracy, whether calculations were required or not.

Improving Diagnostic Efficiency with Frequency Double-Trees and Frequency Nets in Bayesian Reasoning

Background. Medical students often have problems with Bayesian reasoning situations. Representing statistical information as natural frequencies (instead of probabilities) and visualizing them (e.g., with double-trees or net diagrams) leads to higher accuracy in solving these tasks. However, double-trees and net diagrams (which already contain the correct solution of the task, so that the solution could be read of the diagrams) have not yet been studied in medical education. This study examined the influence of information format (probabilities v. frequencies) and visualization (double-tree v. net diagram) on the accuracy and speed of Bayesian judgments. Methods. A total of 142 medical students at different university medical schools (Munich, Kiel, Goettingen, Erlangen, Nuremberg, Berlin, Regensburg) in Germany predicted posterior probabilities in 4 different medical Bayesian reasoning tasks, resulting in a 3-factorial 2 × 2 × 4 design. The diagnostic efficiency for the different versions was represented as the median time divided by the percentage of correct inferences. Results. Frequency visualizations led to a significantly higher accuracy and faster judgments than did probability visualizations. Participants solved 80% of the tasks correctly in the frequency double-tree and the frequency net diagram. Visualizations with probabilities also led to relatively high performance rates: 73% in the probability double-tree and 70% in the probability net diagram. The median time for a correct inference was fastest with the frequency double tree (2:08 min) followed by the frequency net diagram and the probability double-tree (both 2:26 min) and probability net diagram (2:33 min). The type of visualization did not result in a significant difference. Discussion. Frequency double-trees and frequency net diagrams help answer Bayesian tasks more accurately and also more quickly than the respective probability visualizations. Surprisingly, the effect of information format (probabilities v. frequencies) on performance was higher in previous studies: medical students seem also quite capable of identifying the correct solution to the Bayesian task, among other probabilities in the probability visualizations.

Knowledge translation of prediction rules: methods to help health professionals understand their trade-offs

Clinical prediction models are developed with the ultimate aim of improving patient outcomes, and are often turned into prediction rules (e.g. classifying people as low/high risk using cut-points of predicted risk) at some point during the development stage. Prediction rules often have reasonable ability to either rule-in or rule-out disease (or another event), but rarely both. When a prediction model is intended to be used as a prediction rule, conveying its performance using the C-statistic, the most commonly reported model performance measure, does not provide information on the magnitude of the trade-offs. Yet, it is important that these trade-offs are clear, for example, to health professionals who might implement the prediction rule. This can be viewed as a form of knowledge translation. When communicating information on trade-offs to patients and the public there is a large body of evidence that indicates natural frequencies are most easily understood, and one particularly well-received way of depicting the natural frequency information is to use population diagrams. There is also evidence that health professionals benefit from information presented in this way.Here we illustrate how the implications of the trade-offs associated with prediction rules can be more readily appreciated when using natural frequencies. We recommend that the reporting of the performance of prediction rules should (1) present information using natural frequencies across a range of cut-points to inform the choice of plausible cut-points and (2) when the prediction rule is recommended for clinical use at a particular cut-point the implications of the trade-offs are communicated using population diagrams. Using two existing prediction rules, we illustrate how these methods offer a means of effectively and transparently communicating essential information about trade-offs associated with prediction rules.

Which cognitive individual differences predict good Bayesian reasoning? Concurrent comparisons of underlying abilities

We know a lot about how to present Bayesian reasoning tasks in order to aid performance, but less about underlying individual differences that can account for interindividual variability on the same tasks. Such information would be useful for both theoretical and practical reasons. Two theoretical positions, ecological rationality and nested set views, generate multiple hypotheses about which individual difference traits should be most relevant as underlying Bayesian reasoning performance. However, because many of these traits are somewhat overlapping, testing variables in isolation can yield misleading results. The present research assesses Bayesian reasoning abilities in conjunction with multiple individual different measures. Across three experiments, Bayesian reasoning was best predicted by measures of numerical literacy and visuospatial ability, as opposed to several different measures of cognitive thinking dispositions/styles, ability to conceptually model set-theoretic relationships, or cognitive processing ability (working memory span). These results support an ecological rationality view of Bayesian reasoning, rather than nested sets views. There also was some predictive ability for the Cognitive Reflection Task, which was only partially due to the numeracy aspects of that instrument, and further work is needed to clarify if this is a distinct factor. We are now beginning to understand not only how to build Bayesian reasoning tasks, but also how to build good Bayesian reasoners.

Natural frequency trees improve diagnostic efficiency in Bayesian reasoning

When physicians are asked to determine the positive predictive value from the a priori probability of a disease and the sensitivity and false positive rate of a medical test (Bayesian reasoning), it often comes to misjudgments with serious consequences. In daily clinical practice, however, it is not only important that doctors receive a tool with which they can correctly judge-the speed of these judgments is also a crucial factor. In this study, we analyzed accuracy and efficiency in medical Bayesian inferences. In an empirical study we varied information format (probabilities vs. natural frequencies) and visualization (text only vs. tree only) for four contexts. 111 medical students participated in this study by working on four Bayesian tasks with common medical problems. The correctness of their answers was coded and the time spent on task was recorded. The median time for a correct Bayesian inference is fastest in the version with a frequency tree (2:55 min) compared to the version with a probability tree (5:47 min) or to the text only versions based on natural frequencies (4:13 min) or probabilities (9:59 min).The score diagnostic efficiency (calculated by: median time divided by percentage of correct inferences) is best in the version with a frequency tree (4:53 min). Frequency trees allow more accurate and faster judgments. Improving correctness and efficiency in Bayesian tasks might help to decrease overdiagnosis in daily clinical practice, which on the one hand cause cost and on the other hand might endanger patients' safety.

Tversky and Kahneman's Cognitive Illusions: Who Can Solve Them, and Why?

In the present paper we empirically investigate the psychometric properties of some of the most famous statistical and logical cognitive illusions from the "heuristics and biases" research program by Daniel Kahneman and Amos Tversky, who nearly 50 years ago introduced fascinating brain teasers such as the famous Linda problem, the Wason card selection task, and so-called Bayesian reasoning problems (e.g., the mammography task). In the meantime, a great number of articles has been published that empirically examine single cognitive illusions, theoretically explaining people's faulty thinking, or proposing and experimentally implementing measures to foster insight and to make these problems accessible to the human mind. Yet these problems have thus far usually been empirically analyzed on an individual-item level only (e.g., by experimentally comparing participants' performance on various versions of one of these problems). In this paper, by contrast, we examine these illusions as a group and look at the ability to solve them as a psychological construct. Based on an sample of N = 2,643 Luxembourgian school students of age 16-18 we investigate the internal psychometric structure of these illusions (i.e., Are they substantially correlated? Do they form a reflexive or a formative construct?), their connection to related constructs (e.g., Are they distinguishable from intelligence or mathematical competence in a confirmatory factor analysis?), and the question of which of a person's abilities can predict the correct solution of these brain teasers (by means of a regression analysis).

Hypothetical case replacement can be used to quantify the robustness of trial results

OBJECTIVES

We apply a general case replacement framework for quantifying the robustness of causal inferences to characterize the uncertainty of findings from clinical trials.

STUDY DESIGN AND SETTING

We express the robustness of inferences as the amount of data that must be replaced to change the conclusion and relate this to the fragility of trial results used for dichotomous outcomes. We illustrate our approach in the context of an RCT of hydroxychloroquine on pneumonia in COVID-19 patients and a cumulative meta-analysis of the effect of antihypertensive treatments on stroke.

RESULTS

We developed the Robustness of an Inference to Replacement (RIR), which quantifies how many treatment cases with positive outcomes would have to be replaced with hypothetical patients who did not receive a treatment to change an inference. The RIR addresses known limitations of the Fragility Index by accounting for the observed rates of outcomes. It can be used for varying thresholds for inference, including clinical importance.

CONCLUSION

Because the RIR expresses uncertainty in terms of patient experiences, it is more relatable to stakeholders than P-values alone. It helps identify when results are statistically significant, but conclusions are not robust, while considering the rareness of events in the underlying data.

Current Challenges When Using Numbers in Patient Decision Aids: Advanced Concepts

BACKGROUND

Decision aid developers have to convey complex task-specific numeric information in a way that minimizes bias and promotes understanding of the options available within a particular decision. Whereas our companion paper summarizes fundamental issues, this article focuses on more complex, task-specific aspects of presenting numeric information in patient decision aids.

METHODS

As part of the International Patient Decision Aids Standards third evidence update, we gathered an expert panel of 9 international experts who revised and expanded the topics covered in the 2013 review working in groups of 2 to 3 to update the evidence, based on their expertise and targeted searches of the literature. The full panel then reviewed and provided additional revisions, reaching consensus on the final version.

RESULTS

Five of the 10 topics addressed more complex task-specific issues. We found strong evidence for using independent event rates and/or incremental absolute risk differences for the effect size of test and screening outcomes. Simple visual formats can help to reduce common judgment biases and enhance comprehension but can be misleading if not well designed. Graph literacy can moderate the effectiveness of visual formats and hence should be considered in tool design. There is less evidence supporting the inclusion of personalized and interactive risk estimates.

DISCUSSION

More complex numeric information. such as the size of the benefits and harms for decision options, can be better understood by using incremental absolute risk differences alongside well-designed visual formats that consider the graph literacy of the intended audience. More research is needed into when and how to use personalized and/or interactive risk estimates because their complexity and accessibility may affect their feasibility in clinical practice.

Perspectives on the 2 × 2 Matrix: Solving Semantically Distinct Problems Based on a Shared Structure of Binary Contingencies

Cognition is both empowered and limited by representations. The matrix lens model explicates tasks that are based on frequency counts, conditional probabilities, and binary contingencies in a general fashion. Based on a structural analysis of such tasks, the model links several problems and semantic domains and provides a new perspective on representational accounts of cognition that recognizes representational isomorphs as opportunities, rather than as problems. The shared structural construct of a 2 × 2 matrix supports a set of generic tasks and semantic mappings that provide a unifying framework for understanding problems and defining scientific measures. Our model's key explanatory mechanism is the adoption of particular perspectives on a 2 × 2 matrix that categorizes the frequency counts of cases by some condition, treatment, risk, or outcome factor. By the selective steps of filtering, framing, and focusing on specific aspects, the measures used in various semantic domains negotiate distinct trade-offs between abstraction and specialization. As a consequence, the transparent communication of such measures must explicate the perspectives encapsulated in their derivation. To demonstrate the explanatory scope of our model, we use it to clarify theoretical debates on biases and facilitation effects in Bayesian reasoning and to integrate the scientific measures from various semantic domains within a unifying framework. A better understanding of problem structures, representational transparency, and the role of perspectives in the scientific process yields both theoretical insights and practical applications.

Bayesian-Assisted Inference from Visualized Data

A Bayesian view of data interpretation suggests that a visualization user should update their existing beliefs about a parameter's value in accordance with the amount of information about the parameter value captured by the new observations. Extending recent work applying Bayesian models to understand and evaluate belief updating from visualizations, we show how the predictions of Bayesian inference can be used to guide more rational belief updating. We design a Bayesian inference-assisted uncertainty analogy that numerically relates uncertainty in observed data to the user's subjective uncertainty, and a posterior visualization that prescribes how a user should update their beliefs given their prior beliefs and the observed data. In a pre-registered experiment on 4,800 people, we find that when a newly observed data sample is relatively small (N=158), both techniques reliably improve people's Bayesian updating on average compared to the current best practice of visualizing uncertainty in the observed data. For large data samples (N=5208), where people's updated beliefs tend to deviate more strongly from the prescriptions of a Bayesian model, we find evidence that the effectiveness of the two forms of Bayesian assistance may depend on people's proclivity toward trusting the source of the data. We discuss how our results provide insight into individual processes of belief updating and subjective uncertainty, and how understanding these aspects of interpretation paves the way for more sophisticated interactive visualizations for analysis and communication.

Visualizing Uncertainty for Non-Expert End Users: The Challenge of the Deterministic Construal Error

There is a growing body of evidence that numerical uncertainty expressions can be used by non-experts to improve decision quality. Moreover, there is some evidence that similar advantages extend to graphic expressions of uncertainty. However, visualizing uncertainty introduces challenges as well. Here, we discuss key misunderstandings that may arise from uncertainty visualizations, in particular the evidence that users sometimes fail to realize that the graphic depicts uncertainty. Instead they have a tendency to interpret the image as representing some deterministic quantity. We refer to this as the deterministic construal error. Although there is now growing evidence for the deterministic construal error, few studies are designed to detect it directly because they inform participants upfront that the visualization expresses uncertainty. In a natural setting such cues would be absent, perhaps making the deterministic assumption more likely. Here we discuss the psychological roots of this key but underappreciated misunderstanding as well as possible solutions. This is a critical question because it is now clear that members of the public understand that predictions involve uncertainty and have greater trust when uncertainty is included. Moreover, they can understand and use uncertainty predictions to tailor decisions to their own risk tolerance, as long as they are carefully expressed, taking into account the cognitive processes involved.

Numbers, graphs and words - do we really understand the lab test results accessible via the patient portals?

BACKGROUND

The heavy reliance on remote patient care (RPC) during the COVID-19 health crisis may have expedited the emergence of digital health tools that can contribute to safely and effectively moving the locus of care from the hospital to the community. Understanding how laypersons interpret the personal health information accessible to them via electronic patient records (EPRs) is crucial to healthcare planning and the design of services. Yet we still know little about how the format in which personal medical information is presented in the EPR (numerically, verbally, or graphically) affects individuals' understanding of the information, their assessment of its gravity, and the course of action they choose in response.

METHODS

We employed an online questionnaire to assess respondents' reactions to 10 medical decision-making scenarios, where the same information was presented using different formats. In each scenario, respondents were presented with real (anonymized) patient lab results using either numeric expressions, graphs, or verbal expressions. Participants were asked to assess the gravity of the hypothetical patient's condition and the course of action they would follow if they were that patient. The questionnaire was distributed to more than 300 participants, of whom 225 submitted usable responses.

RESULTS

Laypersons were more likely to overestimate the gravity of the information when it was presented either numerically or graphically compared to the narrative format. High perceived gravity was most likely to produce an inclination to actively seek medical attention, even when unwarranted. "Don't know" responses were most likely to produce an inclination to either search the Internet or wait for the doctor to call.

POLICY RECOMMENDATIONS

We discuss the study's implications for the effective design of lab results in the patient portals. We suggest (1) that graphs, tables, and charts would be easier to interpret if coupled with a brief verbal explanation; (2) that highlighting an overall level of urgency may be more helpful than indicating a diversion from the norm; and (3) that statements of results should include the type of follow-up required.

What facilitates Bayesian reasoning? A crucial test of ecological rationality versus nested sets hypotheses

Different theoretical views about Bayesian reasoning (ecological rationality and nested sets views) both claim support from results showing that natural sampling, whole numbers, and pictorial representations help with reasoning performance, although they differ in explaining how those results occur. Three studies (total N = 653) use minimally different numerical presentation formats-varying the singular or plural tense of the context story topic-and presence or absence of an additional icon array picture, to better understand the mechanisms driving these reasoning performance results. Plural wording, indicating a conceptual aggregation (i.e., frequencies) rather than just numerical whole numbers, consistently boosted performance. Icon arrays, in contrast, were helpful only when alongside single-tense information. These results fit more consistently with an ecological rationality view which has long argued that the mind is adapted to work best with frequentist information.

Different Visualizations Cause Different Strategies When Dealing With Bayesian Situations

People often struggle with Bayesian reasoning. However, previous research showed that people's performance (and rationality) can be supported by the way the statistical information is represented. First, research showed that using natural frequencies instead of probabilities as the format of statistical information significantly increases people's performance in Bayesian situations. Second, research also revealed that people's performance increases through using visualization. We have built our paper on existing research in this field. Our main aim was to analyze people's strategies in Bayesian situations that are erroneous even though statistical information is represented as natural frequencies and visualizations. In particular, we compared two pairs of visualization with similar numerical information (tree diagram vs. unit square, and double-tree diagram vs. 2 × 2-table) concerning their impact on people's erroneous strategies in Bayesian situations. For this aim, we conducted an experiment with 540 university students. The students were randomly assigned to four conditions defined by the four different visualizations of statistical information. The students were asked to indicate a fraction in response to four Bayesian situations. We documented the numerator and denominator of the students' responses representing a basic set and a subset in a Bayesian situation. Our results showed that people's erroneous strategies are highly dependent on visualization. A central finding was that the visualization's characteristic of making the nested-sets structure of a Bayesian situation transparent has a facilitating effect on people's Bayesian reasoning. For example, compared to the unit square, a tree diagram does not explicitly visualize the set-subset relations that are relevant in a Bayesian situation. Accordingly, compared to a unit square, a tree diagram partly hinders people in finding the correct denominator in a Bayesian situation, and, in particular, triggers selecting a wrong numerator. By analyzing people's erroneous strategies in Bayesian situations, we contribute to investigating approaches to facilitate Bayesian reasoning and to further develop the teaching of Bayesian reasoning.

The Precision-Bias Distinction for Evaluating Visual Decision Aids for Risk Perception

Risk communication is critically important, for both patients and providers. However, people struggle to understand risks because there are inherent biases and limitations to reasoning under uncertainty. A common strategy to enhance risk communication is the use of decision aids, such as charts or graphs, that depict the risk visually. A problem with prior research on visual decision aids is that it used a metric of performance that confounds 2 underlying constructs: precision and bias. Precision refers to a person's sensitivity to the information, whereas bias refers to a general tendency to overestimate (or underestimate) the level of risk. A visual aid is effective for communicating risk only if it enhances precision or, once precision is suitably high, reduces bias. This article proposes a methodology for evaluating the effectiveness of visual decision aids. Empirical data further illustrate how the new methodology is a significant advancement over more traditional research designs.

Risk literacy assessment of general practitioners and medical students using the Berlin Numeracy Test

Background

The responsibility for helping patients understand potential health benefits and risks, especially regarding screening tests, falls largely to general practitioners (GPs). The Berlin Numeracy Test (BNT) specifically measures risk literacy (i.e., the ability to understand different aspects of statistical numeracy associated with accurate interpretation of information about risks). This study explored the association between risk literacy levels and clinical experience in GPs vs. medical students. Additionally, the effect of GP risk literacy on evaluation of the predictive value of screening tests was examined.

Methods

The participants were 84 GPs and 92 third-year medical students who completed the BNT (total score range 0–4 points). The GPs received an additional case scenario on mammography screening as a simple measure of performance in applying numeracy skills.

Results

Despite having an average of 25.9 years of clinical experience, GPs scored no better than medical students on risk literacy (GPs: 2.33 points, 95% confidence interval [CI] 2.08–2.59; students: 2.34, 95% CI 2.07–2.61; P = .983). Of all GPs, 71.6% (n = 58) greatly overestimated the real predictive value.

Conclusions

In this study, we found no difference in risk literacy between current students and current GPs. GPs lack risk literacy and consequently do not fully understand numeric estimates of probability in routine screening procedures.

A New Visualization for Probabilistic Situations Containing Two Binary Events: The Frequency Net

In teaching statistics in secondary schools and at university, two visualizations are primarily used when situations with two dichotomous characteristics are represented: 2 × 2 tables and tree diagrams. Both visualizations can be depicted either with probabilities or with frequencies. Visualizations with frequencies have been shown to help students significantly more in Bayesian reasoning problems than probability visualizations do. Because tree diagrams or double-trees (which are largely unknown in school) are node-branch structures, these two visualizations (in contrast to the 2 × 2 table) can even simultaneously display probabilities on branches and frequencies inside the nodes. This is a teaching advantage as it allows the frequency concept to be used to better understand probabilities. However, 2 × 2 tables and (double-)trees have a decisive disadvantage: While joint probabilities [e.g., P(A∩B)] are represented in 2 × 2 tables but no conditional probabilities [e.g., P(A|B)], it is exactly the other way around with (double-)trees. Therefore, a visualization that is equally suitable for the representation of joint probabilities and conditional probabilities is desirable. In this article, we present a new visualization—the frequency net—in which all absolute frequencies and all types of probabilities can be depicted. In addition to a detailed theoretical analysis of the frequency net, we report the results of a study with 249 university students that shows that “net diagrams” can improve reasoning without previous instruction to a similar extent as 2 × 2 tables and double-trees. Regarding questions about conditional probabilities, frequency visualizations (2 × 2 table, double-tree, or net diagram with absolute frequencies) are consistently superior to probability visualizations, and the frequency net performs as well as the frequency double-tree. Only the 2 × 2 table with frequencies—the one visualization that participants were already familiar with—led to higher performance rates. If, on the other hand, a question about a joint probability had to be answered, all implemented visualizations clearly supported participants’ performance, but no uniform format effect becomes visible. Here, participants reached the highest performance in the versions with probability 2 × 2 tables and probability net diagrams. Furthermore, after conducting a detailed error analysis, we report interesting error shifts between the two information formats and the different visualizations and give recommendations for teaching probability.

Tackling the outcome bias related to the effectiveness of antibiotics against the common cold: results of a randomized controlled trial applying the Solomon four-group design

In recent years, antimicrobial resistance (AMR) has become an international public health priority. In the area of human medicine, the mis- and overuse of antibiotics is an important contributor to the development of AMR. Such a non-prudent use of antibiotics is especially prevalent in the treatment of viral infections such as the common cold. The present study aims to address the misconception, also known as outcome bias, that antibiotics may be an effective treatment against the common cold by providing a "debiasing" risk communication intervention. It aims at conveying the non-existence of a cause-effect relationship between antibiotics and the reduction of cold-related symptoms through a visual aid and simple explanatory text. A Solomon four-group design was employed to test for within- and between-subjects effects of the intervention as well as potential sensitization effects of the repeated measurement. A total of 311 participants residing in Germany were randomly assigned to receiving (1) a pretest, the debiasing intervention and post-test (2), a pretest, a control stimulus and post-test (3), the debiasing intervention and post-test, or (4) the post-test only. Outcome measures included knowledge about the effectiveness of antibiotics, the attitude toward using it as treatment against the common cold and the evaluation of a scenario describing an irresponsible use of antibiotics. Within-subjects comparisons found that participants receiving the pre- and post-test and intervention showed improved knowledge (t(77) = -2.53, p = .014), attitude (t(77) = -2.09, p = .040), and evaluation measures (t(77) = -2.23, p = .028). The pretest might, however, have caused a sensitization of participants for knowledge-related questions (t(77) = 2.207, p = .029). Between-subjects comparisons found differences in knowledge levels between the post-test only group and both groups receiving the intervention (F(3, 307) = 5.63, p = .001, η2p = .05]. There were no differences between the intervention and control groups with regard to attitude and evaluation of the scenario. While the risk communication intervention led to an increase in knowledge, the outcomes related to attitude and evaluation of a scenario were only affected positively in one group. Therefore, it seems that communication interventions based on visual aids are a simple method to promote the understanding of the true relationship between antibiotic treatment and the decrease of cold-related symptoms. Variables such as attitude and evaluation of a scenario presenting the irresponsible use of antibiotics require, however, additional interventions facilitating a translation of accurate understanding into respective attitudes and judgments.

Graphical Depiction of Statistical Information Improves Gambling-Related Judgments

The domain of gambling is rife with both diagnostic and non-diagnostic information. Previous studies examining scratch card gambling have demonstrated that people are often biased by intuitively appealing, yet non-diagnostic information (i.e., unclaimed prize information). The current study investigated how varying the presentation format of a diagnostic piece of information (i.e., payback percentage) could influence participants’ use of this information when in conflict with unclaimed prize information. We hypothesized that when payback percentage information was presented in a graphical, as opposed to a numerical format, participants would be better at ignoring unclaimed prize information and correspondingly have their preferences become congruent with the true value of the presented scratch cards. In Experiment 1 (N = 201), with payback percentage presented in a numerical format, participants displayed a non-optimal preference for cards with greater numbers of unclaimed prizes and lower payback percentages. This preference was reversed in Experiment 2 (N = 201) when payback percentage was presented in a graphical format. In conclusion, the results of the current study demonstrate how judgments in a scratch card gambling domain can be improved by simply changing the presentation format of a single piece of information.

Fostering the Understanding of Positive Test Results

Background

The majority of health-service users seem unable to properly compute the positive predictive value of medical tests. The research reported in the present study sought to investigate whether, and to what extent, probabilistic inferences about a positive test result can be improved by changing the traditional way in which probability judgments are elicited and medical information is presented.

Methods

Online survey respondents were presented with a positive test result regarding a pregnant woman, and had to estimate the chances that her unborn baby had an anomaly (standard judgment), to apportion the numbers of chances for and against this hypothesis (distributive judgment), and to indicate whether the hypothesis that the baby had an anomaly was more or less likely than its alternative (relative judgment). Test sensitivity and information framing were also manipulated.

Results

Irrespective of education and to some extent of numeracy, the majority of respondents produced correct distributive assessments of chances, which were in line with relative judgments and more accurate than standard ones. When information displayed exclusively positive test results, inferences resulted further improved and unaffected by test sensitivity.

Conclusions

Simple communication strategies that prompt extensional reasoning on the relevant set of number of chances can help individuals to overcome probabilistic inference errors.

Clinical accuracy data presented as natural frequencies improve dentists' caries diagnostic inference: Evidence from a randomized controlled trial

BACKGROUND

The authors assessed whether dentists' diagnostic inferences differ when test accuracy information is communicated using natural frequencies versus conditional probabilities.

METHODS

A parallel, randomized controlled trial with dentists was carried out in Rio de Janeiro, Brazil. The dentists received a question on the probability of a patient having interproximal caries, given a positive bite-wing radiograph. This question was asked using information that was formulated into either natural frequencies or conditional probabilities.

RESULTS

Only 14 (13.9%) of the dentists gave the correct answer; 13 in the natural frequencies group, and 1 in the conditional probabilities group (P < .001). There were 7 nearly correct answers in the natural frequencies group and none in the conditional probabilities group (P = .005).

CONCLUSIONS

Representing diagnostic test accuracy in natural frequencies substantially helped dentists make diagnostic inferences. Nearly twice as many dentists overestimated the presence of interproximal caries when given information in conditional probabilities.

PRACTICAL IMPLICATIONS

Our study findings show information shared using natural frequencies may be more accurately interpreted by dentists than that based on conditional probabilities. Patients will probably receive different standards of care depending on the format in which dentists receive diagnostic test accuracy information.

Benefits and harms of selective oestrogen receptor modulators (SERMs) to reduce breast cancer risk: a cross-sectional study of methods to communicate risk in primary care

BACKGROUND

In Australia, evidence-based guidelines recommend that women consider taking selective oestrogen receptor modulators (SERMs) to reduce their risk of breast cancer. In practice, this requires effective methods for communicating the harms and benefits of taking SERMs so women can make an informed choice.

AIM

To evaluate how different risk presentations influence women's decisions to consider taking SERMs.

DESIGN AND SETTING

Cross-sectional, correlational study of Australian women in general practice.

METHOD

Three risk communication formats were developed that included graphics, numbers, and text to explain the reduction in breast cancer risk and risk of side effects for women taking SERMs (raloxifene or tamoxifen). Women aged 40-74 years in two general practices were shown the risk formats using vignettes of hypothetical women at moderate or high risk of breast cancer and asked to choose 'If this was you, would you consider taking a SERM?' Descriptive statistics and predictors (risk format, level of risk, and type of SERM) of choosing SERMs were determined by logistic regression.

RESULTS

A total of 288 women were recruited (an 88% response rate) between March and May 2017. The risk formats that showed a government statement and an icon array were associated with a greater likelihood of considering SERMs relative to one that showed a novel expected frequency tree. Risk formats for raloxifene and for the high-risk vignettes were also more strongly associated with choosing to consider SERMs. No associations were found with any patient demographics.

CONCLUSION

Specific risk formats may lead to more women considering taking SERMs to reduce breast cancer risk, especially if they are at high risk of the condition. Raloxifene may be a more acceptable SERM to patients.

Biomarker data visualisation for decision making in clinical trials

OBJECTIVE

To understand how visualization of biomarker data is used for decision making in clinical trials, and identify problems with and suggest improvements to this process.

METHODS

We carried out semi-structured interviews with 18 professionals involved in various aspects of developing or using visualizations of biomarker data for decision making in clinical trials. We used an inductive thematic analysis to identify implicit and explicit ideas within the data captured from the interviews.

RESULTS

We identified 6 primary themes, including: how visualizations were used in clinical trials; the importance of having a clear understanding of the underlying data; the purpose or use of the visualization, and the properties of the visualizations themselves. The results show that participants' 'trust' in the visualization depends on access to the underlying data, and that there is currently no standard or straightforward way to support this access.

CONCLUSIONS

Incorporating information about data provenance into biomarker-related visualizations used for decision making in clinical trials may increase users' trust, and therefore facilitate the decision making process.

Cancer screening risk literacy of physicians in training: An experimental study

We investigated what factors may foster or hinder physicians' cancer screening risk literacy-specifically the ability to understand evidence regarding screening effectiveness and make evidence-based recommendations to patients. In an experiment, physicians in training (interns and residents) read statistical information about outcomes from screening for cancer, and had to decide whether to recommend it to a patient. We manipulated the effectiveness of the screening (effective vs. ineffective at reducing mortality) and the demand of the patient to get screened (demand vs. no demand). We assessed participants' comprehension of the presented evidence and recommendation to the patient, as well as a-priori screening beliefs (e.g., that screening is always a good choice), numeracy, science literacy, knowledge of screening statistics, statistical education, and demographics. Stronger positive a-priori screening beliefs, lower knowledge of screening statistics, and lower numeracy were related to worse comprehension of the evidence. Physicians recommended against the ineffective screening but only if they showed good comprehension of the evidence. Physicians' recommendations were further based on the perceived benefits from screening but not on perceived harms, nor the patient's demands. The current study demonstrates that comprehension of cancer screening statistics and the ability to infer the potential benefits for patients are essential for evidence-based recommendations. However, strong beliefs in favor of screening fostered by promotion campaigns may influence how physicians evaluate evidence about specific screenings. Fostering physician numeracy skills could help counteract such biases and provide evidence-based recommendations to patients.

An Eye-Tracking Study of Statistical Reasoning With Tree Diagrams and 2 × 2 Tables

Changing the information format from probabilities into frequencies as well as employing appropriate visualizations such as tree diagrams or 2 × 2 tables are important tools that can facilitate people's statistical reasoning. Previous studies have shown that despite their widespread use in statistical textbooks, both of those visualization types are only of restricted help when they are provided with probabilities, but that they can foster insight when presented with frequencies instead. In the present study, we attempt to replicate this effect and also examine, by the method of eye tracking, why probabilistic 2 × 2 tables and tree diagrams do not facilitate reasoning with regard to Bayesian inferences (i.e., determining what errors occur and whether they can be explained by scan paths), and why the same visualizations are of great help to an individual when they are combined with frequencies. All ten inferences of N = 24 participants were based solely on tree diagrams or 2 × 2 tables that presented either the famous "mammography context" or an "economics context" (without additional textual wording). We first asked participants for marginal, conjoint, and (non-inverted) conditional probabilities (or frequencies), followed by related Bayesian tasks. While solution rates were higher for natural frequency questions as compared to probability versions, eye-tracking analyses indeed yielded noticeable differences regarding eye movements between correct and incorrect solutions. For instance, heat maps (aggregated scan paths) of distinct results differed remarkably, thereby making correct and faulty strategies visible in the line of theoretical classifications. Moreover, the inherent structure of 2 × 2 tables seems to help participants avoid certain Bayesian mistakes (e.g., "Fisherian" error) while tree diagrams seem to help steer them away from others (e.g., "joint occurrence"). We will discuss resulting educational consequences at the end of the paper.

Numeracy and Risk Literacy: What Have We Learned so Far?

Numerical skills are essential to make informed decisions in our daily life. Unfortunately, many people lack basic numeracy, which limits their ability to accurately interpret risks (i.e., risk literacy). In this paper, we provide an overview of research investigating the role of numeracy in two prominent domains, where most research was concentrated, health and finance. We summarize what has been learned so far in these domains and suggest promising venues for future research. We conclude that it is important to conduct interventions to improve numeracy in less numerate individuals and to help them make informed decisions and achieve better life outcomes.

Aiding Risk Information learning through Simulated Experience (ARISE): A Comparison of the Communication of Screening Test Information in Explicit and Simulated Experience Formats

OBJECTIVE

To determine whether the use of Aiding Risk Information learning through Simulated Experience (ARISE) to communicate conditional probabilities about maternal serum screening results for Down syndrome promotes more accurate positive predictive value (PPV) estimates and conceptual understanding of screening, compared with explicitly providing individuals with this information via numerical summary or icon array.

METHOD

In experiment 1, 582 participants completed an online study in which they were asked to estimate the PPV and rate their attitudes toward a screening test when information was presented in either a description (required calculation of the PPV), explicit (PPV was provided and had to be identified), or an ARISE format (PPV was inferred through experience-based learning). In experiment 2, 316 participants estimated the PPV and rated their attitudes toward screening based on information presented in either an icon array (identify the icons that represent the PPV) or ARISE format.

RESULTS

In experiment 1, ARISE elicited the most accurate PPV estimates compared with the description and explicit formats, and both the explicit and ARISE formats led to more unfavorable attitudes toward screening. In experiment 2, both the icon array and ARISE resulted in similar PPV estimates; however, ARISE led to more negative attitudes toward screening.

CONCLUSIONS

These findings suggest that ARISE may be superior to other formats in the communication of PPV information for screening tests. However, differences in the complexity of the formats vary and require further investigation.

How to Improve Performance in Bayesian Inference Tasks: A Comparison of Five Visualizations

Bayes’ formula is a fundamental statistical method for inference judgments in uncertain situations used by both laymen and professionals. However, since people often fail in situations where Bayes’ formula can be applied, how to improve their performance in Bayesian situations is a crucial question. We based our research on a widely accepted beneficial strategy in Bayesian situations, representing the statistical information in the form of natural frequencies. In addition to this numerical format, we used five visualizations: a 2 × 2-table, a unit square, an icon array, a tree diagram, and a double-tree diagram. In an experiment with 688 undergraduate students, we empirically investigated the effectiveness of three graphical properties of visualizations: area-proportionality, use of discrete and countable statistical entities, and graphical transparency of the nested-sets structure. We found no additional beneficial effect of area proportionality. In contrast, the representation of discrete objects seems to be beneficial. Furthermore, our results show a strong facilitating effect of making the nested-sets structure of a Bayesian situation graphically transparent. Our results contribute to answering the questions of how and why a visualization could facilitate judgment and decision making in situations of uncertainty.

ATOM: A Grammar for Unit Visualizations

Unit visualizations are a family of visualizations where every data item is represented by a unique visual mark-a visual unit-during visual encoding. For certain datasets and tasks, unit visualizations can provide more information, better match the user's mental model, and enable novel interactions compared to traditional aggregated visualizations. Current visualization grammars cannot fully describe the unit visualization family. In this paper, we characterize the design space of unit visualizations to derive a grammar that can express them. The resulting grammar is called ATOM, and is based on passing data through a series of layout operations that divide the output of previous operations recursively until the size and position of every data point can be determined. We evaluate the expressive power of the grammar by both using it to describe existing unit visualizations, as well as to suggest new unit visualizations.