Natural frequency trees improve diagnostic efficiency in Bayesian reasoning

Bayesian versus diagnostic information in physician-patient communication: Effects of direction of statistical information and presentation of visualization

BACKGROUND

Communicating well with patients is a competence central to everyday clinical practice, and communicating statistical information, especially in Bayesian reasoning tasks, can be challenging. In Bayesian reasoning tasks, information can be communicated in two different ways (which we call directions of information): The direction of Bayesian information (e.g., proportion of people tested positive among those with the disease) and the direction of diagnostic information (e.g., the proportion of people having the disease among those tested positive). The purpose of this study was to analyze the impact of both the direction of the information presented and whether a visualization (frequency net) is presented with it on patient's ability to quantify a positive predictive value.

MATERIAL AND METHODS

109 participants completed four different medical cases (2⨯2⨯4 design) that were presented in a video; a physician communicated frequencies using different directions of information (Bayesian information vs. diagnostic information). In half of the cases for each direction, participants were given a frequency net. After watching the video, participants stated a positive predictive value. Accuracy and speed of response were analyzed.

RESULTS

Communicating with Bayesian information led to participant performance of only 10% (without frequency net) and 37% (with frequency net) accuracy. The tasks communicated with diagnostic information but without a frequency net were correctly solved by 72% of participants, but accuracy rate decreased to 61% when participants were given a frequency net. Participants with correct responses in the Bayesian information version without visualization took longest to complete the tasks (median of 106 seconds; median of 13.5, 14.0, and 14.5 seconds in other versions).

DISCUSSION

Communicating with diagnostic information rather than Bayesian information helps patients to understand specific information better and more quickly. Patients' understanding of the relevance of test results is strongly dependent on the way the information is presented.

The Use of Visualizations to Improve Bayesian Reasoning: A Literature Review

Decisions are often made under uncertainty. The most that one can do is use prior knowledge (e.g., base rates, prior probabilities, etc.) and make the most probable choice given the information we have. Unfortunately, most people struggle with Bayesian reasoning. Poor performance within Bayesian reasoning problems has led researchers to investigate ways to improve Bayesian reasoning. Many have found success in using natural frequencies instead of probabilities to frame problems. Beyond the quantitative format, there is growing literature on the use of visualizations or visual representations to improve Bayesian reasoning, which will be the focus of this review. In this review, we discuss studies that have found visualizations to be effective for improving Bayesian reasoning in a lab or classroom setting and discuss the considerations for using visualizations, paying special attention to individual differences. In addition, we will review the factors that influence Bayesian reasoning, such as natural frequencies vs. probabilities, problem format, individual differences, and interactivity. We also provide general and specific suggestions for future research.

How to Train Novices in Bayesian Reasoning

Bayesian Reasoning is both a fundamental idea of probability and a key model in applied sciences for evaluating situations of uncertainty. Bayesian Reasoning may be defined as the dealing with, and understanding of, Bayesian situations. This includes various aspects such as calculating a conditional probability (performance), assessing the effects of changes to the parameters of a formula on the result (covariation) and adequately interpreting and explaining the results of a formula (communication). Bayesian Reasoning is crucial in several non-mathematical disciplines such as medicine and law. However, even experts from these domains struggle to reason in a Bayesian manner. Therefore, it is desirable to develop a training course for this specific audience regarding the different aspects of Bayesian Reasoning. In this paper, we present an evidence-based development of such training courses by considering relevant prior research on successful strategies for Bayesian Reasoning (e.g., natural frequencies and adequate visualizations) and on the 4C/ID model as a promising instructional approach. The results of a formative evaluation are described, which show that students from the target audience (i.e., medicine or law) increased their Bayesian Reasoning skills and found taking part in the training courses to be relevant and fruitful for their professional expertise.

Improving Diagnostic Efficiency with Frequency Double-Trees and Frequency Nets in Bayesian Reasoning

Background. Medical students often have problems with Bayesian reasoning situations. Representing statistical information as natural frequencies (instead of probabilities) and visualizing them (e.g., with double-trees or net diagrams) leads to higher accuracy in solving these tasks. However, double-trees and net diagrams (which already contain the correct solution of the task, so that the solution could be read of the diagrams) have not yet been studied in medical education. This study examined the influence of information format (probabilities v. frequencies) and visualization (double-tree v. net diagram) on the accuracy and speed of Bayesian judgments. Methods. A total of 142 medical students at different university medical schools (Munich, Kiel, Goettingen, Erlangen, Nuremberg, Berlin, Regensburg) in Germany predicted posterior probabilities in 4 different medical Bayesian reasoning tasks, resulting in a 3-factorial 2 × 2 × 4 design. The diagnostic efficiency for the different versions was represented as the median time divided by the percentage of correct inferences. Results. Frequency visualizations led to a significantly higher accuracy and faster judgments than did probability visualizations. Participants solved 80% of the tasks correctly in the frequency double-tree and the frequency net diagram. Visualizations with probabilities also led to relatively high performance rates: 73% in the probability double-tree and 70% in the probability net diagram. The median time for a correct inference was fastest with the frequency double tree (2:08 min) followed by the frequency net diagram and the probability double-tree (both 2:26 min) and probability net diagram (2:33 min). The type of visualization did not result in a significant difference. Discussion. Frequency double-trees and frequency net diagrams help answer Bayesian tasks more accurately and also more quickly than the respective probability visualizations. Surprisingly, the effect of information format (probabilities v. frequencies) on performance was higher in previous studies: medical students seem also quite capable of identifying the correct solution to the Bayesian task, among other probabilities in the probability visualizations.