Dyadic decisions with numerical and verbal probabilities

Eye-tracking evidence for fixation asymmetries in verbal and numerical quantifier processing

When people are given quantified information (e.g., ‘there is a 60% chance of rain’), the format of quantifiers (i.e., numerical: ‘a 60% chance’ vs. verbal: ‘it is likely’) might affect their decisions. Previous studies with indirect cues of judgements and decisions (e.g., response times, decision outcomes) give inconsistent findings that could support either a more intuitive process for verbal than numerical quantifiers or a greater focus on the context (e.g., rain) for verbal than numerical quantifiers. We used two pre-registered eye-tracking experiments (n(1) = 148, n(2) = 133) to investigate decision-making processes with verbal and numerical quantifiers. Participants evaluated multiple verbally or numerically quantified nutrition labels (Experiment 1) and weather forecasts (Experiment 2) with different context valence (positive or negative), and quantities (‘low’, ‘medium’, or ‘high’ in Experiment 1 and ‘possible’, ‘likely’, or ‘very likely’ in Experiment 2) presented in a fully within-subjects design. Participants looked longer at verbal than numerical quantifiers, and longer at the contextual information with verbal quantifiers. Quantifier format also affected judgements and decisions: in Experiment 1, participants judged positive labels to be better in the verbal compared to the equivalent numerical condition (and to be worse for negative labels). In Experiment 2, participants decided on rain protection more for a verbal forecast of rain than the equivalent numerical forecast. The results fit the explanation that verbal quantifiers put more focus on the informational context than do numerical quantifiers, rather than prompting more intuitive decisions.

No Effect of Cue Format on Automation Dependence in an Aided Signal Detection Task

OBJECTIVE

To investigate whether manipulating the format of an automated decision aid's cues can improve participants' information integration strategies in a signal detection task.

BACKGROUND

Automation-aided decision making is often suboptimal, falling well short of statistically ideal levels. The choice of format in which the cues from the aid are displayed may help users to better understand and integrate the aid's judgments with their own.

METHOD

Participants performed a signal detection task that asked them to classify random dot images as either blue or orange dominant. They made their judgments either unaided or with assistance from a 93% reliable automated decision aid. The aid provided a binary judgment, along with an estimate of signal strength in the form of either a raw value, a likelihood ratio, or a confidence rating (Experiments 1 and 2) or a binary judgment along with either a verbal or verbal-visuospatial expression of confidence (Experiment 3). Aided sensitivity was benchmarked to the predictions of various statistical models of collaborative decision making.

RESULTS AND CONCLUSION

Aided performance was suboptimal, matching the predictions of some of the least efficient models. Most importantly, performance was similar across cue formats.

APPLICATION

Results indicate that changes to the format in which cues from a signal detection aid are rendered are unlikely to dramatically improve the efficiency of automation-aided decision making.

Communicative Functions of Directional Verbal Probabilities: Speaker's Choice, Listener's Inference, and Reference Points

Verbal probabilities have directional communicative functions, and most can be categorized as positive (e.g., “it is likely”) or negative (e.g., “it is doubtful”). We examined the communicative functions of verbal probabilities based on the reference point hypothesis According to this hypothesis, listeners are sensitive to and can infer a speaker's reference points based on the speaker's selected directionality. In four experiments (two of which examined speakers’ choice of directionality and two of which examined listeners’ inferences about a speaker's reference point), we found that listeners could make inferences about speakers’ reference points based on the stated directionality of verbal probability. Thus, the directionality of verbal probabilities serves the communicative function of conveying information about a speaker's reference point.

How fast can you (possibly) do it, or how long will it (certainly) take? Communicating uncertain estimates of performance time

Recent research on verbal probability statements has revealed that some expressions (e.g., possible) are especially appropriate for describing outcomes in the high end of a distribution, whereas other expressions (e.g., certain) are more appropriate for describing low-end values. However, some dimensions appear to be reversible, with higher achievements sometimes associated with high and sometimes with low values, depending on frame. We report three experiments where this "reframing effect" is studied in communications of estimated performance time, both from a speaker's and from a listener's perspective. We hypothesize that statements describing tasks as "taking time" suggest a duration frame, and find accordingly that statements about how many hours that possibly will be spent on a task, or the time a task possibly takes, lead to high time estimates. Statements focusing on the actor's role suggest, in contrast, a speed frame, thus statements about what the actor can possibly do lead to low time estimates. Estimates of the time a task certainly takes or when it is certainly done follow the opposite pattern. The results are in line with approaches that see production and comprehension of language as a dynamical and context-driven process.

How incidental values from the environment affect decisions about money, risk, and delay

How different are £0.50 and £1.50, “a small chance” and “a good chance,” or “three months” and “nine months”? Our studies show that people behave as if the differences between these values are altered by incidental everyday experiences. Preference for a £1.50 lottery rather than a £0.50 lottery was stronger among individuals exposed to intermediate supermarket prices than among those exposed to lower or higher prices. Preference for “a good chance” rather than “a small chance” of winning a lottery was stronger among participants who predicted intermediate probabilities of rain than among those who predicted lower or higher chances of rain. Preference for consumption in “three months” rather than “nine months” was stronger among participants who planned for an intermediate birthday than among participants who planned for a sooner or later birthday. These fluctuations directly challenge economic accounts that translate monies, risks, and delays into subjective equivalents with stable functions. The decision-by-sampling model—in which subjective values are rank positions constructed from comparisons with samples—predicts these effects and indicates a primary role for sampling in decision making.

Human Reliance on an Automated Combat ID System: Effects of Display Format

Informing users of reliability levels can engender more appropriate reliance on automated systems. In an experiment, reliability information was presented on a rifle-mounted interface for an automated combat identification (CID) system to participants in a first-person shooter simulation of a combat environment. Results showed two effects: First, a mesh display supported better discrimination of hostile and friendly targets than did a pie chart. Second, participants shown an integrated display were better able to adjust their reliance to the changing reliability level than participants shown separated displays. Providing reliability information changed participants' reliance strategies; however, display integration appears to play a role in how system reliability information is used.

A review of human linguistic probability processing: General principles and empirical evidence

This article reviews research on how people use and understand linguistic expressions of uncertainty, with a view toward the needs of researchers and others interested in artificial intelligence systems. We discuss and present empirical results within an inductively developed theoretical framework consisting of two background assumptions and six principles describing the underlying cognitive processes.

Improving Communication of Uncertainty in the Reports of the Intergovernmental Panel on Climate Change

The Intergovernmental Panel on Climate Change (IPCC) assesses information relevant to the understanding of climate change and explores options for adaptation and mitigation. The IPCC reports communicate uncertainty by using a set of probability terms accompanied by global interpretational guidelines. The judgment literature indicates that there are large differences in the way people understand such phrases, and that their use may lead to confusion and errors in communication. We conducted an experiment in which subjects read sentences from the 2007 IPCC report and assigned numerical values to the probability terms. The respondents' judgments deviated significantly from the IPCC guidelines, even when the respondents had access to these guidelines. These results suggest that the method used by the IPCC is likely to convey levels of imprecision that are too high. We propose an alternative form of communicating uncertainty, illustrate its effectiveness, and suggest several additional ways to improve the communication of uncertainty.

Health Literacy and Cancer Risk Perception: Implications for Genomic Risk Communication

Background. As new genomic technology expands the number of medical tests available to physicians and patients, identifying gaps in our understanding of how best to communicate risk is increasingly important. We examined how health literacy informs breast cancer survivors' understanding of and meaning assigned to recurrence risks yielded by genomic tests. Methods. Study participants were posttreatment female breast cancer survivors (N =163 ) recruited at a university breast cancer clinic. We assessed their health literacy (using REALM) and their interpretation of hypothetical recurrence risk results from a genomic test, presented in several verbal and numerical formats. Analyses controlled for women's objective recurrence risk, age, income, and race. Results. Women with lower health literacy gave higher mean estimates of recurrence risk for a hypothetical group of women with early-stage breast cancer than did women with higher health literacy (52% v. 30%, P < 0:001 ). Women with lower health literacy also gave more variable estimates in this and several other tasks. When making chemotherapy decisions using risks presented in verbal formats, decisions by women with lower health literacy were less sensitive to the difference between low and high recurrence risk. Ease of understanding of risk formats differed by health literacy. Conclusions. Health literacy affected the meanings women assigned to recurrence risk when presented in certain formats. The greater variability in responding by women with lower health literacy supports the hypothesis that they have less precise mental representations of risk, but more research is needed to rule out other possible explanations.

Interpersonal comparison of subjective probabilities: Toward translating linguistic probabilities

Interpersonal variability in understanding linguistic probabilities can adversely affect decision making. Using the fact that everyone judges canonical probability events similarly in a manner consistent with axiom systems that yield a probability measure, we developed and tested a method for comparing the meanings of probability phrases across individuals. An experiment demonstrated that despite extreme heterogeneity in participants' linguistic probability lexicons, interpersonal similarity in phrase meaning is well predicted by phrase rank order within the lexicons. Thus, equally ranked phrases have similar meanings, and individual differences in linguistic probabilities may simply be explained by the phrases people use at each rank.

Displaying uncertainty: investigating the effects of display format and specificity

We conducted four studies regarding the representation of probabilistic information. Experiments 1 through 3 compared performance on a simulated stock purchase task, in which information regarding stock profitability was probabilistic. Two variables were manipulated: display format for probabilistic information (blurred and colored icons, linguistic phrases, numeric expressions, and combinations) and specificity level (in which the number and size of discrete steps into which the probabilistic information was mapped differed). Results indicated few performance differences attributable to display format; however, performance did improve with greater specificity. Experiment 4, in which participants generated membership functions corresponding to three display formats, found a high degree of similarity in functions across formats and participants and a strong relationship between the shape of the membership function and the intended meaning of the representation. These results indicate that participants can successfully interpret nonnumeric representations of uncertainty and can use such representations in a manner similar to the way numeric expressions are used in a decision-making task. Actual or potential applications of this research include the use of graphical representations of uncertainty in systems such as command and control and situation displays.

You Say "Probable" and I Say "Likely": Improving Interpersonal Communication With Verbal Probability Phrases

When forecasters and decision makers describe uncertain events using verbal probability terms, there is a risk of miscommunication because people use different probability phrases and interpret them in different ways. In an effort to facilitate the communication process, the authors investigated various ways of converting the forecasters' verbal probabilities to the decision maker's terms. The authors present 3 studies in which participants judged the probabilities of distinct events using both numerical and verbal probabilities. They define several indexes of interindividual coassignment of phrases to the same events and evaluate the conversion methods by comparing the values of these indexes for the converted and the unconverted judgments. In all the cases studied, the conversion methods significantly reduced the error rates in communicating uncertainties.

Making trade-offs: a probabilistic and context-sensitive model of choice behavior

The stochastic difference model assumes that decision makers trade normalized attribute value differences when making choices. The model is stochastic, with choice probabilities depending on the normalized difference variable, d, and a decision threshold, delta. The decision threshold indexes a person's sensitivity to attribute value differences and is a free estimated parameter of the model. Depending on the choice context, a person may be more or less sensitive to attribute value differences, and hence delta may be used to measure context effects. With proportional difference used as the normalization, the proportional difference model (PD) was tested with 9 data sets, including published data (e.g., J. L. Myers, M. M. Suydam, & B. Gambino, 1965; A. Tversky, 1969). The model accounted for individual and group data well and described violations of stochastic dominance, independence, and weak and strong stochastic transitivity.

The Directionality of Verbal Probability Expressions: Effects on Decisions, Predictions, and Probabilistic Reasoning

Verbal phrases denoting uncertainty are of two kinds: positive, suggesting the occurrence of a target outcome, and negative, drawing attention to its nonoccurrence (Teigen & Brun, 1995). This directionality is correlated with, but not identical to, high and low p values. Choice of phrase will in turn influence predictions and decisions. A treatment described as having "some possibility" of success will be recommended, as opposed to when it is described as "quite uncertain," even if the probability of cure referred to by these two expressions is judged to be the same (Experiment 1). Individuals who formulate their chances of achieving a successful outcome in positive terms are supposed to make different decisions than individuals who use equivalent, but negatively formulated, phrases (Experiments 2 and 3). Finally, negative phrases lead to fewer conjunction errors in probabilistic reasoning than do positive phrases (Experiment 4). For instance, a combination of 2 "uncertain" outcomes is readily seen to be "very uncertain." But positive phrases lead to fewer disjunction errors than do negative phrases. Thus verbal probabilistic phrases differ from numerical probabilities not primarily by being more "vague," but by suggesting more clearly the kind of inferences that should be drawn. Copyright 1999 Academic Press.

Design and validation of a bedside decision instrument to elicit a patient's preference concerning allogenic bone marrow transplantation in chronic myeloid leukemia

The objective of this study was to design and validate a bedside decision instrument to be used by patients with chronic myeloid leukemia and their physicians in deciding between the therapeutic alternatives of bone marrow transplantation and conservative management during the early phase of disease. A decision board was constructed containing detailed scenarios associated with the treatment alternatives, together with estimates of survival probabilities at various periods of followup. The instrument was tested on 42 healthy hospital personnel and validated by measuring the extent to which systematic alterations in the scenarios with respect to toxicities and survival probabilities produced predicted shifts in treatment preferences. A subgroup of respondents was randomized to receive information through the decision board alone or a shorter and less informative version of the instrument, followed by the decision board. The direction and strength of stated preferences were compared, together with satisfaction for these preferences. The direction and strength of preferences between bone marrow transplantation or conservative chemotherapy were influenced in a predictable way by changes in the toxicity and survival descriptions in the scenarios. Using the test-retest method in 16 subjects, the stated preferences were found to be highly reliable (intraclass correlation coefficient, 0.87). The mean level of satisfaction with the stated preference, on a scale from not at all satisfied = 1 to very satisfied = 5, was higher for those exposed to the decision board (3.7, SD 1.06) compared with those presented with the short version (2.95, SD 0.67) (P < 0.01). The results demonstrate the feasibility and acceptability of the instrument in healthy individuals. The preferences elicited by the instrument appear to be reliable and valid according to prespecified constructs of the relation between the information provided and the preferences predicted. These results support further testing of this approach in actual patients.

The roles of experience and domain of expertise in using numerical and verbal probability terms in medical decisions

Verbal probability terms are frequently used in medical practice. In the present experiment the use of verbal and numerical probability terms in medical decisions was investigated. Interns, residents in surgery and internal medicine, surgeons, and internists were asked to make treatment decisions for three different cases (acute appendicitis, angina pectoris, and an imaginary disease) and were also asked to give numerical interpretations of a series of verbal probability terms. In the second stage of the experiment the respondents received the same cases, but with numerical probability terms. The results showed no effect of context or of domain experience on the interpretation of verbal terms. Residents and experienced surgeons more often agreed on treatment decisions when chance information was presented in numerical terms as compared with verbal terms. Physicians were less confident when verbal terms were presented, but only for the less familiar decision problems. Finally, physicians turned out to be better in Bayesian reasoning when numerical terms were used. Experienced physicians were quite accurate in estimating the posterior probability in the appendicitis case, but not in the imaginary-disease case.