Validation of the Subjective Numeracy Scale: effects of low numeracy on comprehension of risk communications and utility elicitations

Valuation of health states in the US study to establish disability weights: lessons from the literature

The metric of disability-adjusted life years (DALYs) has become the global standard of measuring burden of disease. DALYs are comprised of years of life lost due to premature mortality and years of healthy life lost due to living with disability. In order to calculate the second part of the DALY equation, disease specific disability weights have to be established, i.e. measures for the decline of health associated with these disease states, which vary between zero for perfect health and one for death. Although these disability weights are key for estimating DALYs, there have not been many comprehensive studies with empirical determinations of them. This article describes a systematic review on the state of the art with respect to empirically determining disability weights. Based on this review, a multi-method approach is outlined, which has also been implemented in a US study to measure burden of disease. This approach involves the use of psychometric methodology as well as economic trade-off methods for determining the value of health states. It is conceptualized as a disaggregated approach, where the disability weight of any health state can be calculated if the attributes of this health state are known. The US study received the collaboration of experts from more than 20 institutes of the National Institutes of Health and of the Centers for Disease Control and Prevention. First results will be available by the end of this year.

The role of numeracy in informed consent for surveys

If empirical estimates of disclosure risk are included in informed consent statements for surveys or other forms of research, participants must be able to understand the information provided. Using data from an online vignette-based experiment, this article explores the role that numeracy or quantitative literacy may play in comprehension of disclosure risk. Results suggest that less numerate persons are less sensitive to extreme differences in the disclosure risk described in the hypothetical vignettes.

Socioeconomic status, statistical confidence, and patient-provider communication: an analysis of the Health Information National Trends Survey (HINTS 2007)

The increasing trend of exposing patients seeking health advice to numerical information has the potential to adversely impact patient-provider relationships especially among individuals with low literacy and numeracy skills. We used the HINTS 2007 to provide the first large scale study linking statistical confidence (as a marker of subjective numeracy) to demographic variables and a health-related outcome (in this case the quality of patient-provider interactions). A cohort of 7,674 individuals answered sociodemographic questions, a question on how confident they were in understanding medical statistics, a question on preferences for words or numbers in risk communication, and a measure of patient-provider interaction quality. Over thirty-seven percent (37.4%) of individuals lacked confidence in their ability to understand medical statistics. This was particularly prevalent among the elderly, low income, low education, and non-White ethnic minority groups. Individuals who lacked statistical confidence demonstrated clear preferences for having risk-based information presented with words rather than numbers and were 67% more likely to experience a poor patient-provider interaction, after controlling for gender, ethnicity, insurance status, the presence of a regular health care professional, and the language of the telephone interview. We will discuss the implications of our findings for health care professionals.

How numeracy influences risk comprehension and medical decision making

We review the growing literature on health numeracy, the ability to understand and use numerical information, and its relation to cognition, health behaviors, and medical outcomes. Despite the surfeit of health information from commercial and noncommercial sources, national and international surveys show that many people lack basic numerical skills that are essential to maintain their health and make informed medical decisions. Low numeracy distorts perceptions of risks and benefits of screening, reduces medication compliance, impedes access to treatments, impairs risk communication (limiting prevention efforts among the most vulnerable), and, based on the scant research conducted on outcomes, appears to adversely affect medical outcomes. Low numeracy is also associated with greater susceptibility to extraneous factors (i.e., factors that do not change the objective numerical information). That is, low numeracy increases susceptibility to effects of mood or how information is presented (e.g., as frequencies vs. percentages) and to biases in judgment and decision making (e.g., framing and ratio bias effects). Much of this research is not grounded in empirically supported theories of numeracy or mathematical cognition, which are crucial for designing evidence-based policies and interventions that are effective in reducing risk and improving medical decision making. To address this gap, we outline four theoretical approaches (psychophysical, computational, standard dual-process, and fuzzy trace theory), review their implications for numeracy, and point to avenues for future research.

Understanding the role of numeracy in health: proposed theoretical framework and practical insights

Numeracy-that is, how facile people are with mathematical concepts and their applications-is gaining importance in medical decision making and risk communication. This article proposes six critical functions of health numeracy. These functions are integrated into a theoretical framework on health numeracy that has implications for risk communication and medical decision-making processes. The authors examine practical underpinnings for targeted interventions aimed at improving such processes as a function of health numeracy. They hope that the proposed functions and theoretical framework will spur more research to determine how an understanding of health numeracy can lead to more effective communication and decision outcomes.

Effect of risk ladder format on risk perception in high- and low-numerate individuals

Utilizing a random sample from the general population (N= 257), we examined the effect of the radon risk ladder on risk perception, as qualified by respondents' numeracy. The radon risk ladder provides comparative risk information about the radon equivalent of smoking risk. We compared a risk ladder providing smoking risk information with a risk ladder not providing this information. A 2 (numeracy; high, low) x 3 (risk level; high, medium, low) x 2 (smoking risk comparison: with/without) between subjects experimental design was used. A significant (p < 0.045) three-way interaction between format, risk level, and numeracy was identified. Participants with low numeracy skills, as well as participants with high numeracy skills, generally distinguished between low, medium, and high risk levels when the risk ladder with comparative smoking risk information was presented. When the risk ladder without the comparative information about the smoking risk was presented, low-numerate individuals differentiated between risk levels to a much lesser extent than high-numerate individuals did. These results provide empirical evidence that the risk ladder can be a useful tool in enabling people to interpret various risk levels. Additionally, these results allow us to conclude that providing comparative information within a risk ladder is particularly helpful to the understanding of different risk levels by people with low numeracy skills.

What is my cancer risk? How internet-based cancer risk assessment tools communicate individualized risk estimates to the public: content analysis

Background

Internet-based cancer risk assessment tools have the potential to inform the public about cancer risk and promote risk-reducing behaviors. However, poorly communicated information on these websites may result in unintended adverse health outcomes.

Objective

This study examined whether: (1) Internet-based cancer risk assessment tools use risk communication formats that facilitate comprehension and reduce bias (as identified by the empirical literature); (2) the use of these formats varies by website affiliation; and (3) the websites provided information necessary to evaluate the quality of the risk estimate.

Methods

A content analysis of Internet-based cancer risk assessment tools was conducted. The terms calculate cancer risk, cancer risk calculator, estimate cancer risk, assess cancer risk, and cancer risk assessment were searched using three search engines. We identified 47 risk assessment tools and coded each according to standardized criteria. We calculated simple frequencies on all coding categories and performed crosstabulations but did not conduct formal statistical analysis due to small cell sizes.

Results

Use of risk communication formats that facilitate comprehension and reduce bias varied widely (eg, 30% of websites [14/47] provided absolute and comparative risk information but 83% [39/47] provided safety messages). Use of formats that facilitate comprehension varied by website affiliation and communication strategy (eg, only 8.3% [1/12] websites affiliated with the health care industry provided absolute and comparative risk information, but 83% [5/6] of websites affiliated with a governmental organization did so). Only 53% (25/47) of websites provided information about the statistical model or the peer-reviewed literature that was used to calculate the risk estimate.

Conclusion

Internet-based cancer risk assessment tools varied in their use of risk communication formats that facilitate comprehension and reduce bias. Formats that are difficult to understand may cause people to misperceive their cancer risk and consequently take inappropriate action.

Effect of Risk Communication Formats on Risk Perception Depending on Numeracy

Objective. To examine the influence of numeracy on interpreting various risk communication formats. Design. A random sample of women (N = 266) completed a questionnaire assessing numeracy and risk perception of prenatal test results and results of colon cancer screening tests. The authors examined the relationships between risk level (high v. low) and format of risk presentation (ratio, pictogram, or Paling Perspective Scale) and whether these relationships differed based on the numeracy skills of the participant. Results. The authors identified a significant (P <0.001) 3-way interaction between format, risk level, and numeracy: high-numerate participants in the low-risk group perceived the test results as less risky compared with participants in the high-risk group (P < 0.001) with the Paling Perspective Scale but not with the other formats. For low-numerate participants, they did not observe differences between low- and high-risk scenarios for any of the 3 formats. The results were similar for the Down syndrome and colon cancer scenarios. Overall, the pictogram resulted in significantly lower risk ratings compared with the Paling Perspective Scale and the ratio with numerator 1 (P < 0.001). Conclusion. Different communication formats may produce different risk perceptions, but the effect is qualified by patients' numeracy skills.

Laypersons' responses to the communication of uncertainty regarding cancer risk estimates

OBJECTIVE

To explore laypersons' responses to the communication of uncertainty associated with individualized cancer risk estimates and to identify reasons for individual differences in these responses.

DESIGN

A qualitative study was conducted using focus groups. Participants were informed about a new colorectal cancer risk prediction model, and presented with hypothetical individualized risk estimates using presentation formats varying in expressed uncertainty (range v. point estimate). Semistructured interviews explored participants' responses to this information.

PARTICIPANTS AND SETTING

Eight focus groups were conducted with 48 adults aged 50 to 74 residing in 2 major US metropolitan areas, Chicago, IL and Washington, DC. Purposive sampling was used to recruit participants with a high school or greater education, some familiarity with information technology, and no personal or immediate family history of cancer.

RESULTS

Participants identified several sources of uncertainty regarding cancer risk estimates, including missing data, limitations in accuracy and source credibility, and conflicting information. In comparing presentation formats, most participants reported greater worry and perceived risk with the range than with the point estimate, consistent with the phenomenon of "ambiguity aversion.'' However, others reported the opposite effect or else indifference between formats. Reasons suggested by participants' responses included individual differences in optimism and motivations to reduce feelings of vulnerability and personal lack of control. Perceptions of source credibility and risk mutability emerged as potential mediating factors.

CONCLUSIONS

Laypersons' responses to the communication of uncertainty regarding cancer risk estimates differ, and include both heightened and diminished risk perceptions. These differences may be attributable to personality, cognitive, and motivational factors.

Ethical considerations in psychiatric genetics

Ethical considerations in psychiatric genetics are highly complex and fluid. This review introduces the reader to the wide range of ethical considerations in this field by examining four characteristics of genetic information. First, genetic information may, to a greater or lesser extent, predict a person's future health. Second, learning about one's genotype may have profound psychosocial consequences. Third, genetic information pertains to a person's biological relatives and thus can affect family members, communities, and population groups. Finally, psychiatric genetics is a rapidly evolving field. None of these characteristics is necessarily "exceptional" or unique to genetics, but they provide a useful framework for teasing apart a complex set of ethical considerations. This article reviews conceptual and empirical data that speak to these four characteristics and then presents a set of conceptual frameworks that can be used to systematically analyze the ethics of psychiatric genetic research and clinical genotyping. Finally, directions for future study are described--including the urgent need to gather data on actual risks and benefits of psychiatric genetic research and clinical applications, so that their utility can be assessed and appropriate ethical safeguards identified.

Conceptual problems in laypersons' understanding of individualized cancer risk: a qualitative study

OBJECTIVE

To explore laypersons' understanding of individualized cancer risk estimates, and to identify conceptual problems that may limit this understanding.

BACKGROUND

Risk prediction models are increasingly used to provide people with information about their individual risk of cancer and other diseases. However, laypersons may have difficulty understanding individualized risk information, because of conceptual as well as computational problems.

DESIGN

A qualitative study was conducted using focus groups. Semi-structured interviews explored participants' understandings of the concept of risk, and their interpretations of a hypothetical individualized colorectal cancer risk estimate.

SETTING AND PARTICIPANTS

Eight focus groups were conducted with 48 adults aged 50-74 years residing in two major US metropolitan areas. Participants had high school or greater education, some familiarity with information technology, and no personal or family history of cancer.

RESULTS

Several important conceptual problems were identified. Most participants thought of risk not as a neutral statistical concept, but as signifying danger and emotional threat, and viewed cancer risk in terms of concrete risk factors rather than mathematical probabilities. Participants had difficulty acknowledging uncertainty implicit to the concept of risk, and judging the numerical significance of individualized risk estimates. The most challenging conceptual problems related to conflict between subjective and objective understandings of risk, and difficulties translating aggregate-level objective risk estimates to the individual level.

CONCLUSIONS

Several conceptual problems limit laypersons' understanding of individualized cancer risk information. These problems have implications for future research on health numeracy, and for the application of risk prediction models in clinical and public health settings.

Improving understanding of adjuvant therapy options by using simpler risk graphics

BACKGROUND

To help oncologists and breast cancer patients make informed decisions about adjuvant therapies, online tools such as Adjuvant! provide tailored estimates of mortality and recurrence risks. However, the graphical format used to display these results (a set of 4 horizontal stacked bars) may be suboptimal. The authors tested whether using simpler formats would improve comprehension of the relevant risk statistics.

METHODS

A total of 1,619 women, aged 40-74 years, completed an Internet-administered survey vignette about adjuvant therapy decisions for a patient with an estrogen receptor-positive tumor. Participants were randomized to view 1 of 4 risk graphics, a base version that mirrored the Adjuvant! format, an alternate graph that showed only 2 options (those that included hormonal therapy), a graph that used a pictograph format, or a graph that included both changes. Outcome measures included comprehension of key statistics, time required to complete the task, and graph-perception ratings.

RESULTS

The simplifying format changes significantly improved comprehension, especially when both changes were implemented together. Compared with participants who viewed the base 4-option bar graph, respondents who, instead, viewed a 2-option pictograph version were more accurate when they reported the incremental risk reduction achievable from adding chemotherapy to hormonal therapy (77% vs 51%; P< .001), answered that question more quickly (median time, 28 seconds vs 42 seconds; P< .001), and liked the graph more (mean, 7.67 vs 6.88; P< .001).

CONCLUSIONS

Although most patients will only view risk calculators such as Adjuvant! in consultation with their clinicians, simplifying design graphics could significantly improve patients' comprehension of statistics essential for informed decision making about adjuvant therapies.

The impact of the format of graphical presentation on health-related knowledge and treatment choices

OBJECTIVE

To evaluate the ability of six graph formats to impart knowledge about treatment risks/benefits to low and high numeracy individuals.

METHODS

Participants were randomized to receive numerical information about the risks and benefits of a hypothetical medical treatment in one of six graph formats. Each described the benefits of taking one of two drugs, as well as the risks of experiencing side effects. Main outcome variables were verbatim (specific numerical) and gist (general impression) knowledge. Participants were also asked to rate their perceptions of the graphical format and to choose a treatment.

RESULTS

2412 participants completed the survey. Viewing a pictograph was associated with adequate levels of both types of knowledge, especially for lower numeracy individuals. Viewing tables was associated with a higher likelihood of having adequate verbatim knowledge vs. other formats (p<0.001) but lower likelihood of having adequate gist knowledge (p<0.05). All formats were positively received, but pictograph was trusted by both high and low numeracy respondents. Verbatim and gist knowledge were significantly (p<0.01) associated with making a medically superior treatment choice.

CONCLUSION

Pictographs are the best format for communicating probabilistic information to patients in shared decision making environments, particularly among lower numeracy individuals.

PRACTICE IMPLICATIONS

Providers can consider using pictographs to communicate risk and benefit information to patients of different numeracy levels.

Communicating side effect risks in a tamoxifen prophylaxis decision aid: the debiasing influence of pictographs

OBJECTIVE

To experimentally test whether using pictographs (image matrices), incremental risk formats, and varied risk denominators would influence perceptions and comprehension of side effect risks in an online decision aid about prophylactic use of tamoxifen to prevent primary breast cancers.

METHODS

We recruited 631 women with elevated breast cancer risk from two healthcare organizations. Participants saw tailored estimates of the risks of 5 side effects: endometrial cancer, blood clotting, cataracts, hormonal symptoms, and sexual problems. Presentation format was randomly varied in a three factor design: (A) risk information was displayed either in pictographs or numeric text; (B) presentations either reported total risks with and without tamoxifen or highlighted the incremental risk most relevant for decision making; and (C) risk estimates used 100 or 1000 person denominators. Primary outcome measures included risk perceptions and gist knowledge.

RESULTS

Incremental risk formats consistently lowered perceived risk of side effects but resulted in low knowledge when displayed by numeric text only. Adding pictographs, however, produced significantly higher comprehension levels.

CONCLUSIONS

Pictographs make risk statistics easier to interpret, reducing biases associated with incremental risk presentations.

PRACTICE IMPLICATIONS

Including graphs in risk communications is essential to support an informed treatment decision-making process.

Clinical implications of numeracy: theory and practice

BACKGROUND

Low numeracy is pervasive and constrains informed patient choice, reduces medication compliance, limits access to treatments, impairs risk communication, and affects medical outcomes; therefore, it is incumbent upon providers to minimize its adverse effects.

PURPOSE

We provide an overview of research on health numeracy and discuss its implications in clinical contexts.

CONCLUSIONS

Low numeracy cannot be reliably inferred on the basis of patients' education, intelligence, or other observable characteristics. Objective and subjective assessments of numeracy are available in short forms and could be used to tailor health communication. Low scorers on these assessments are subject to cognitive biases, irrelevant cues (e.g., mood), and sharper temporal discounting. Because prevention of the leading causes of death (e.g., cancer and cardiovascular disease) depends on taking action now to prevent serious consequences later, those low in numeracy are likely to require more explanation of risk to engage in prevention behaviors. Visual displays can be used to make numerical relations more transparent, and different types of displays have different effects (e.g., greater risk avoidance). Ironically, superior quantitative processing seems to be achieved by focusing on qualitative gist and affective meaning, which has important implications for empowering patients to take advantage of the evidence in evidence-based medicine.

Measuring numeracy without a math test: development of the Subjective Numeracy Scale

BACKGROUND

Basic numeracy skills are necessary before patients can understand the risks of medical treatments. Previous research has used objective measures, similar to mathematics tests, to evaluate numeracy.

OBJECTIVES

To design a subjective measure (i.e., self-assessment) of quantitative ability that distinguishes low- and high-numerate individuals yet is less aversive, quicker to administer, and more usable for telephone and Internet surveys than existing numeracy measures.

RESEARCH DESIGN

Paper-and-pencil questionnaires.

SUBJECTS

The general public (N = 703) surveyed at 2 hospitals.

MEASURES

Forty-nine subjective numeracy questions were compared to measures of objective numeracy.

RESULTS

An 8-item measure, the Subjective Numeracy Scale (SNS), was developed through several rounds of testing. Four items measure people's beliefs about their skill in performing various mathematical operations, and 4 measure people's preferences regarding the presentation of numerical information. The SNS was significantly correlated with Lipkus and others' objective numeracy scale (correlations: 0.63-0.68) yet was completed in less time (24 s/item v. 31 s/item, P < 0.05) and was perceived as less stressful (1.62 v. 2.69, P < 0.01) and less frustrating (1.92 v. 2.88, P < 0.01). Fifty percent of participants who completed the SNS volunteered to participate in another study, whereas only 8% of those who completed the Lipkus and others scale similarly volunteered (odds ratio = 11.00, 95% confidence interval = 2.14-56.65).

CONCLUSIONS

The SNS correlates well with mathematical test measures of objective numeracy but can be administered in less time and with less burden. In addition, it is much more likely to leave participants willing to participate in additional research and shows much lower rates of missing or incomplete data.