The 1-in-X effect on the subjective assessment of medical probabilities

Leveraging Math Cognition to Combat Health Innumeracy

Rational numbers (i.e., fractions, percentages, decimals, and whole-number frequencies) are notoriously difficult mathematical constructs. Yet correctly interpreting rational numbers is imperative for understanding health statistics, such as gauging the likelihood of side effects from a medication. Several pernicious biases affect health decision-making involving rational numbers. In our novel developmental framework, the natural-number bias-a tendency to misapply knowledge about natural numbers to all numbers-is the mechanism underlying other biases that shape health decision-making. Natural-number bias occurs when people automatically process natural-number magnitudes and disregard ratio magnitudes. Math-cognition researchers have identified individual differences and environmental factors underlying natural-number bias and devised ways to teach people how to avoid these biases. Although effective interventions from other areas of research can help adults evaluate numerical health information, they circumvent the core issue: people's penchant to automatically process natural-number magnitudes and disregard ratio magnitudes. We describe the origins of natural-number bias and how researchers may harness the bias to improve rational-number understanding and ameliorate innumeracy in real-world contexts, including health. We recommend modifications to formal math education to help children learn the connections among natural and rational numbers. We also call on researchers to consider individual differences people bring to health decision-making contexts and how measures from math cognition might identify those who would benefit most from support when interpreting health statistics. Investigating innumeracy with an interdisciplinary lens could advance understanding of innumeracy in theoretically meaningful and practical ways.

Frequency or total number? A comparison of different presentation formats on risk perception during COVID-19

Curbing the COVID-19 pandemic remains an ongoing global challenge. Institutions often release information about confirmed COVID-19 cases by citing the total number of cases (e.g., 100,000), their (relative) frequency (e.g., 100 per 1,000,000), or occasionally their proportion (e.g., 0.0001) in a region. I compared the effect of these three presentation formats — total cases, frequency, and proportion — on people’s perceived risk. I found people perceived a higher risk of COVID-19 from a total-cases format than from frequency formats when the denominators are relatively small, and the lowest risk from a proportion format. Correspondingly, people underestimated total infections when given frequency and overestimated frequency when given total number of cases. Additional comparisons were made among mathematically equivalent variations of frequency formats (e.g., 1 in 100, 10 in 1,000, 1,000 in 10,000, etc.). The results provided qualified support for denominator neglect, which seems to occur in bins into which denominators are grouped (e.g., 1–1000, 10000–100000), such that only across bins could participants perceive differences. Finally, a mixed format of proportion and total cases reduced perceived risks from total cases alone, while a mixed format of frequency and total cases failed to produce similar results. I conclude by providing concrete suggestions regarding COVID-19 information releases.

Effective communication regarding risk of fracture for individuals at risk of fragility fracture: a scoping review

Two scoping reviews were conducted to review recommendations and guidelines for communication regarding general health risk, and to investigate communication strategies regarding risk of fracture. Healthcare professionals are invited to apply these recommendations to optimize a patient-centered approach to reducing risk of fracture.

INTRODUCTION

To conduct a scoping review of the medical literature regarding recommendations and tools for effective communication between healthcare professionals and patients regarding general health risk and risk of fracture.

METHODS

The scoping review was divided into two parts to search for (1) studies presenting recommendations and guidelines for communication regarding general health risk; (2) studies investigating communication regarding risk of fracture for individuals at risk for fractures. Medline was searched in April 2020 to identify relevant studies.

RESULTS

The scoping review included 43 studies on communication with regard to general health risk and 25 studies about communication regarding risk of fracture. Recommendations for effective communication with regard to risk are presented. Communication of numeric data on risk should be adapted to the literacy and numeracy levels of the individual patient. Patient understanding of numerical data can be enhanced with appropriate use of visual aids (e.g., pie charts, icon arrays, bar charts, pictograms). The FRAX® tool is the most recommended and most used tool for assessing risk of fracture. Communication sent as individualized letters to patients following DXA scans has been studied, although patient understanding of their risk of fracture is often reported as low using this technique. Use of visual aids may improve patient understanding.

CONCLUSION

Healthcare professionals are encouraged to apply recommendations presented in this scoping review in their clinical practice. Patient understanding of risk of fracture should be confirmed by making sure that patients feel free to ask questions and express their concerns. This will contribute to an optimal patient-centered approach. Developing online tools to convert the probability of fracture into patient-friendly visual presentations could facilitate communication between healthcare professionals and patients about risk of fracture.

Communicating personalized risks from COVID-19: guidelines from an empirical study

As increasing amounts of data accumulate on the effects of the novel coronavirus SARS-CoV-2 and the risk factors that lead to poor outcomes, it is possible to produce personalized estimates of the risks faced by groups of people with different characteristics. The challenge of how to communicate these then becomes apparent. Based on empirical work (total n = 5520, UK) supported by in-person interviews with the public and physicians, we make recommendations on the presentation of such information. These include: using predominantly percentages when communicating the absolute risk, but also providing, for balance, a format which conveys a contrasting (higher) perception of risk (expected frequency out of 10 000); using a visual linear scale cut at an appropriate point to illustrate the maximum risk, explained through an illustrative 'persona' who might face that highest level of risk; and providing context to the absolute risk through presenting a range of other 'personas' illustrating people who would face risks of a wide range of different levels. These 'personas' should have their major risk factors (age, existing health conditions) described. By contrast, giving people absolute likelihoods of other risks they face in an attempt to add context was considered less helpful. We note that observed effect sizes generally were small. However, even small effects are meaningful and relevant when scaled up to population levels.

Current Best Practice for Presenting Probabilities in Patient Decision Aids: Fundamental Principles

BACKGROUND

Shared decision making requires evidence to be conveyed to the patient in a way they can easily understand and compare. Patient decision aids facilitate this process. This article reviews the current evidence for how to present numerical probabilities within patient decision aids.

METHODS

Following the 2013 review method, we assembled a group of 9 international experts on risk communication across Australia, Germany, the Netherlands, the United Kingdom, and the United States. We expanded the topics covered in the first review to reflect emerging areas of research. Groups of 2 to 3 authors reviewed the relevant literature based on their expertise and wrote each section before review by the full authorship team.

RESULTS

Of 10 topics identified, we present 5 fundamental issues in this article. Although some topics resulted in clear guidance (presenting the chance an event will occur, addressing numerical skills), other topics (context/evaluative labels, conveying uncertainty, risk over time) continue to have evolving knowledge bases. We recommend presenting numbers over a set time period with a clear denominator, using consistent formats between outcomes and interventions to enable unbiased comparisons, and interpreting the numbers for the reader to meet the needs of varying numeracy.

DISCUSSION

Understanding how different numerical formats can bias risk perception will help decision aid developers communicate risks in a balanced, comprehensible manner and avoid accidental "nudging" toward a particular option. Decisions between probability formats need to consider the available evidence and user skills. The review may be useful for other areas of science communication in which unbiased presentation of probabilities is important.

How is cervical cancer screening information communicated in UK websites? Cross-sectional analysis of content and quantitative presentation formats

OBJECTIVES

To investigate whether UK websites about cervical cancer screening targeted to the public include (1) information about benefits and risks of screening, possible screening results and cervical cancer statistics, (2) quantitative presentation formats recommended in the risk communication literature and (3) appeals for participation and/or informed decision-making.

DESIGN

Cross-sectional analysis of websites using a comprehensive checklist of information items on screening benefits, risks, possible results and cervical cancer statistics.

OUTCOME MEASURES

We recorded the number of websites that contained each of the information items, and the presentation format used for probabilistic information (no quantification provided, verbal quantifiers only, different types of numerical formats and/or graphs). We also recorded the number of websites containing appeals for participation and/or informed decision-making.

SETTING

Websites were identified through the most common Google search terms used in the UK to find information on cervical screening, according to GoogleTrends and a commercial internet-monitoring programme. Two additional websites were identified by the authors as relevant.

RESULTS

After applying exclusion criteria, 14 websites were evaluated, including websites of public and private health service providers, charities, a medical society and a pharmacy. The websites mentioned different benefits, risks of screening and possible results. However, specific content varied between websites. Probabilistic information was often presented using non-recommended formats, including relative risk reductions to express screening benefits, and verbal quantifiers without numbers to express risks. Appeals for participation were present in most websites, with almost half also mentioning informed decision-making.

CONCLUSIONS

UK websites about cervical cancer screening were generally balanced. However, benefits and risks were presented using different formats, potentially hindering comparisons. Additionally, recommendations from the literature to facilitate understanding of quantitative information and facilitate informed decisions were often not followed. Designing websites that adhere to existing recommendations may support informed screening uptake.

Ratio Format Shapes Health Decisions: The Practical Significance of the "1-in-X" Effect

Prior research found that "1-in-X" ratios led to higher and less accurate subjective probability than "N-in-X*N" ratios or other formats, even though they featured the same mathematical information. It is unclear, however, whether the effect transfers into health decisions, and the practical significance of the effect is undetermined. Based on previous findings and risk communication theories, we hypothesized that the 1-in-X effect would occur and transfer into relevant decisions. We also tested whether age, gender, and education differences would moderate the 1-in-X effect on decision making. We conducted 3 well-powered experiments ( n = 1912) using a sample from the general adult UK population to test our hypotheses, estimated the effect, and excluded a possible methodological explanation for such a transfer. In hypothetical scenarios, participants decided whether to travel to Kenya given the chance of contracting malaria (experiment 1) and whether to take recommended steroids given the side effects (experiments 2 and 3). Across the experiments, we replicated a small to medium 1-in-X effect on the perceived probability (Hedge's g = -0.36; 95% confidence interval [CI], -0.47 to -0.24; z = -6.18; P < 0.001) and found a small effect on subsequent decisions (odds ratio = 1.32; 95% CI, 1.10-1.59; z = 2.99; P = 0.003). The perceived probability fully mediated the effect of the ratio format on decision. Age, gender, and education did not moderate the 1-in-X effect on decision. We argue that a high prevalence of 1-in-X ratios in medical communication makes these small changes clinically relevant. Therefore, to communicate information accurately, 1-in-X ratios should not be used or at least used cautiously in medical communication.

Best Methods of Communicating Clinical Trial Data to Improve Understanding of Treatments for Patients with Multiple Sclerosis

BACKGROUND

Patients' understanding of treatment risks and benefits is a prerequisite for shared decision making. Yet, patients with multiple sclerosis (MS) do not accurately understand treatment information provided in regular clinical consultations.

OBJECTIVES

To identify the best methods of communicating clinical trial data to improve the understanding of treatments among patients with MS and to also examine the relationship between patients' understanding with decisional conflict, individual traits, and MS symptoms.

METHODS

A repeated-measures study was used. A sample of relapsing-remitting patients with MS was recruited from National Health Service sites in the United Kingdom. Patients were presented with hypothetical treatment risks and benefits from faux clinical trials. Treatments were communicated using absolute terms, relative terms, and numbers needed to treat/harm. The presence of baseline information with each method was also manipulated. Patients' understanding and conflict in treatment decisions were assessed. Individual traits and MS symptoms were also recorded.

RESULTS

Understanding was better when treatments were communicated in absolute terms (mean 3.99 ± 0.93) compared with relative terms (mean 2.93 ± 0.91; P < 0.001) and numbers needed to treat/harm (mean 2.89 ± 0.88; P < 0.001). Adding baseline information to all methods significantly improved understanding (mean 5.04 ± 0.96) compared with no baseline information (mean 1.50 ± 0.74; P < 0.001). Understanding was not related to conflict in treatment decisions (r = -0.131; P = 0.391). Numeracy, IQ, and cognitive impairments were significantly related to patients' understanding of treatments.

CONCLUSIONS

Treatment risks and benefits should ideally be communicated using absolute terms, alongside baseline information. Patients with MS with low numeracy, low IQ, and reduced cognitive skills should be supported during treatment education.

The effect of different graphical and numerical likelihood formats on perception of likelihood and choice

BACKGROUND

Quantitative risk information plays an important role in decision making about health. This study focuses on commonly used numerical and graphical formats and examines their effect on perception of different likelihoods and choice preferences.

METHODS

An experimental study was conducted with 192 participants, who evaluated 2 sets of 4 lotteries. Numerical formats to describe likelihood varied systematically between participants (X%, X-in-100, or 1-in-X). The effect of graphic formats (bar charts, icon charts) was assessed as a within-subjects factor. Dependent measures included perceived likelihood, choice preferences about participating in the lottery, and processing times.

RESULTS

Numerical likelihoods presented as 1-in-X were processed fastest and were perceived as conveying larger likelihoods than the X-in-100 and percentages formats (mean response times in seconds: 5.65 v. 7.31 and 6.50; mean rating on a 1-9 scale: 4.38 v. 3.30 and 3.31, respectively). The 1-in-X format also evoked a stronger willingness to participate in a lottery than the 2 other numerical formats. The effect of adding graphs on perceived likelihood was moderated by numerical aptitude. Graphs reduced ratings of perceived likelihood of participants with lower numeracy, while there was no overall effect for participants with higher numeracy.

CONCLUSION

Perception of likelihood differs significantly depending on the numerical format used. The 1-in-X format yields higher perceived likelihoods and it appears to be the easiest format to interpret. Graphs primarily affect perception of likelihood of people with lower numerical aptitude. These effects should be taken into account when discussing medical risks with patients.

The effect of prognostic data presentation format on perceived risk among surrogate decision makers of critically ill patients: a randomized comparative trial

PURPOSE

The purpose of this study is to determine whether varying the format used to present prognostic data alters the perception of risk among surrogate decision makers in the intensive care unit (ICU).

METHODS

This was a prospective randomized comparative trial conducted in a 23-bed adult tertiary ICU. Enrolled surrogate decision makers were randomized to 1 of 2 questionnaires, which presented hypothetical ICU scenarios, identical other than the format in which prognostic data were presented (eg, frequencies vs percentages). Participants were asked to rate the risk associated with each prognostic statement.

RESULTS

We enrolled 141 surrogate decision makers. The perception of risk varied significantly dependent on the presentation format. For "quantitative data," risks were consistently perceived as higher, when presented as frequencies (eg, 1 in 50) compared with equivalent percentages (eg, 2%). Framing "qualitative data" in terms of chance of "death" rather than "survival" led to a statistically significant increase in perceived risks. Framing "quantitative" data in this way did not significantly affect risk perception.

CONCLUSION

Data format had a significant effect on how surrogate decision makers interpreted risk. Qualitative statements are interpreted widely and affected by framing. Where possible, multiple quantitative formats should be used for presenting prognostic information.

Decisive Evidence on a Smaller-Than-You-Think Phenomenon

Accurate perception of medical probabilities communicated to patients is a cornerstone of informed decision making. People, however, are prone to biases in probability perception. Recently, Pighin and others extended the list of such biases with evidence that “1-in-X” ratios (e.g., “1 in 12”) led to greater perceived probability and worry about health outcomes than “N-in-X*N” ratios (e.g., “10 in 120”). Subsequently, the recommendation was to avoid using “1-in-X” ratios when communicating probabilistic information to patients. To warrant such a recommendation, we conducted 5 well-powered replications and synthesized the available data. We found that 3 out of the 5 replications yielded statistically nonsignificant findings. In addition, our results showed that the “1-in-X” effect was not moderated by numeracy, cognitive reflection, age, or gender. To quantify the evidence for the effect, we conducted a Bayes factor meta-analysis and a traditional meta-analysis of our 5 studies and those of Pighin and others (11 comparisons, N = 1131). The meta-analytical Bayes factor, which allowed assessment of the evidence for the null hypothesis, was very low, providing decisive evidence to support the existence of the “1-in-X” effect. The traditional meta-analysis showed that the overall effect was significant (Hedges’ g = 0.42, 95% CI 0.29–0.54). Overall, we provide decisive evidence for the existence of the “1-in-X” effect but suggest that it is smaller than previously estimated. Theoretical and practical implications are discussed.

Presenting quantitative information about decision outcomes: a risk communication primer for patient decision aid developers

BACKGROUND

Making evidence-based decisions often requires comparison of two or more options. Research-based evidence may exist which quantifies how likely the outcomes are for each option. Understanding these numeric estimates improves patients' risk perception and leads to better informed decision making. This paper summarises current "best practices" in communication of evidence-based numeric outcomes for developers of patient decision aids (PtDAs) and other health communication tools.

METHOD

An expert consensus group of fourteen researchers from North America, Europe, and Australasia identified eleven main issues in risk communication. Two experts for each issue wrote a "state of the art" summary of best evidence, drawing on the PtDA, health, psychological, and broader scientific literature. In addition, commonly used terms were defined and a set of guiding principles and key messages derived from the results.

RESULTS

The eleven key components of risk communication were: 1) Presenting the chance an event will occur; 2) Presenting changes in numeric outcomes; 3) Outcome estimates for test and screening decisions; 4) Numeric estimates in context and with evaluative labels; 5) Conveying uncertainty; 6) Visual formats; 7) Tailoring estimates; 8) Formats for understanding outcomes over time; 9) Narrative methods for conveying the chance of an event; 10) Important skills for understanding numerical estimates; and 11) Interactive web-based formats. Guiding principles from the evidence summaries advise that risk communication formats should reflect the task required of the user, should always define a relevant reference class (i.e., denominator) over time, should aim to use a consistent format throughout documents, should avoid "1 in x" formats and variable denominators, consider the magnitude of numbers used and the possibility of format bias, and should take into account the numeracy and graph literacy of the audience.

CONCLUSION

A substantial and rapidly expanding evidence base exists for risk communication. Developers of tools to facilitate evidence-based decision making should apply these principles to improve the quality of risk communication in practice.

Playing the numbers: how hepatitis C patients create meaning and make healthcare decisions from medical test results

In this article we describe how patients assign meanings to medical test results and use these meanings to justify their actions. Evidence is presented from lay interpretations of medical tests for monitoring hepatitis C viral infection (HCV) to show how numeracy becomes embodied in the absence of physical symptoms. Illness narratives from 307 individuals infected with HCV were collected from the internet and analysed qualitatively. As part of standard medical care, chronically infected HCV patients are required to have periodic blood tests for laboratory testing. The lab results are presented numerically and compared with established physiological standards. HCV patients' knowledge and interpretations of test results have important consequences for their health behaviour and their medical decisions. In their stories, the patients described their decisions to begin, delay or stop treatment and developed strategies to alter their diet, exercise and use alternative therapies according to changes in their test result. The perceived meanings of test results are powerful signifiers that are capable of altering the course of HCV patients' illness, lives and stories. An interpretive model of health numeracy has the advantage of promoting understanding between patients and healthcare providers over a model that views innumeracy as a skill deficit.

Using Comparison Scenarios to Improve Prenatal Risk Communication

The present research provides empirical evidence of whether communicating the prenatal risk of chromosomal anomalies using comparison scenarios influences women’s ability to distinguish between different risk levels. In 2 experiments, participants read a description of a hypothetical woman who was learning of the risk of chromosomal anomaly as a result of a prenatal screening test. Both experiments used a 3 (risk level) × 3 (scenario) full between-subjects design. In accordance with the experimental condition, participants were presented with a low (e.g., 1 in 5390), a medium (e.g., 1 in 770), or a high risk value (e.g., 1 in 110). Such risk values were presented either on their own or along with additional information illustrating a comparison scenario that provided 2 numerical comparison points. Participants were asked to evaluate the risk of chromosomal anomaly. In Experiment 2, participants’ numeracy skills were also assessed. Results showed that the use of comparison scenarios results in significant differences in perceived risk across risk levels whereas such differences are not significant without the comparison scenario, but such a technique has differential effects according to participants’ capacity to deal with numbers. Although the technique is beneficial for high-numerate participants, it has no effect on low-numerate participants.