Impact of presentation of research results on likelihood of prescribing medications to patients with left ventricular dysfunction

The Science of Visual Data Communication: What Works

Effectively designed data visualizations allow viewers to use their powerful visual systems to understand patterns in data across science, education, health, and public policy. But ineffectively designed visualizations can cause confusion, misunderstanding, or even distrust-especially among viewers with low graphical literacy. We review research-backed guidelines for creating effective and intuitive visualizations oriented toward communicating data to students, coworkers, and the general public. We describe how the visual system can quickly extract broad statistics from a display, whereas poorly designed displays can lead to misperceptions and illusions. Extracting global statistics is fast, but comparing between subsets of values is slow. Effective graphics avoid taxing working memory, guide attention, and respect familiar conventions. Data visualizations can play a critical role in teaching and communication, provided that designers tailor those visualizations to their audience.

Prescribers' Knowledge and Skills for Interpreting Research Results: A Systematic Review

INTRODUCTION

Appropriate medication prescribing may be influenced by a prescriber's ability to understand and interpret medical research. The objective of this review was to synthesize the research related to prescribers' critical appraisal knowledge and skills-defined as the understanding of statistical methods, biases in studies, and relevance and validity of evidence.

METHODS

We searched PubMed and other databases from January 1990 through September 2015. Two reviewers independently screened and selected studies of any design conducted in the United States, the United Kingdom, or Canada that involved prescribers and that objectively measured critical appraisal knowledge, skills, understanding, attitudes, or prescribing behaviors. Data were narratively synthesized.

RESULTS

We screened 1204 abstracts, 72 full-text articles, and included 29 studies. Study populations included physicians. Physicians' extant knowledge and skills were in the low to middle of the possible score ranges and demonstrated modest increases in response to interventions. Physicians with formal education in epidemiology, biostatistics, and research demonstrated higher levels of knowledge and skills. In hypothetical scenarios presenting equivalent effect sizes, the use of relative effect measures was associated with greater perceptions of medication effectiveness and intent to prescribe, compared with the use of absolute effect measures. The evidence was limited by convenience samples and study designs that limit internal validity.

DISCUSSION

Critical appraisal knowledge and skills are limited among physicians. The effect measure used can influence perceptions of treatment effectiveness and intent to prescribe. How critical appraisal knowledge and skills fit among the myriad of influences on prescribing behavior is not known.

Effects of presenting risk information in different formats to cardiologists. A Latin American survey

OBJECTIVE

Previous publications demonstrated that the presentation of treatment benefits in terms of relative risk reduction (RRR) rather than in terms of absolute risk reduction (ARR) or number of patients to treat (NNT) might favor the perception of outcome effectiveness. The objective was to perform a cognitive evaluation to assess how the manner in which risks and benefits of screening methods and treatments are presented can affect medical care decision-taking in a sample of cardiologists.

METHODS

Four-hundred and six Latin American cardiologists answered a questionnaire reporting the results of clinical trials presented as RRR, ARR or NNT, associated or not to biased graphs.

RESULTS

Cardiologists' decision-taking was different when comparing treatment benefits presented as RRR (62.2%) vs. ARR (40.4%) (p=0.000000), and as RRR vs. NNT (44.4%) (p=0.000000). However, their decision-taking was similar when information was presented as NNT or ARR (p=0.073). The inclusion of biased graphs was misinterpreted as an actual data difference (RRR: 61.6% vs. ARR: 14.0%, p=0.000000).

CONCLUSIONS

This study demonstrated that Latin American cardiologists could misinterpret statistical data when information of clinical trials is presented in terms of RRR. We emphasize the need to enhance cardiologists' training in quantitative techniques, to improve medical care decision-making.

Cognitive biases and heuristics in medical decision making: a critical review using a systematic search strategy

BACKGROUND

The role of cognitive biases and heuristics in medical decision making is of growing interest. The purpose of this study was to determine whether studies on cognitive biases and heuristics in medical decision making are based on actual or hypothetical decisions and are conducted with populations that are representative of those who typically make the medical decision; to categorize the types of cognitive biases and heuristics found and whether they are found in patients or in medical personnel; and to critically review the studies based on standard methodological quality criteria.

METHOD

Data sources were original, peer-reviewed, empirical studies on cognitive biases and heuristics in medical decision making found in Ovid Medline, PsycINFO, and the CINAHL databases published in 1980-2013. Predefined exclusion criteria were used to identify 213 studies. During data extraction, information was collected on type of bias or heuristic studied, respondent population, decision type, study type (actual or hypothetical), study method, and study conclusion.

RESULTS

Of the 213 studies analyzed, 164 (77%) were based on hypothetical vignettes, and 175 (82%) were conducted with representative populations. Nineteen types of cognitive biases and heuristics were found. Only 34% of studies (n = 73) investigated medical personnel, and 68% (n = 145) confirmed the presence of a bias or heuristic. Each methodological quality criterion was satisfied by more than 50% of the studies, except for sample size and validated instruments/questions. Limitations are that existing terms were used to inform search terms, and study inclusion criteria focused strictly on decision making.

CONCLUSIONS

Most of the studies on biases and heuristics in medical decision making are based on hypothetical vignettes, raising concerns about applicability of these findings to actual decision making. Biases and heuristics have been underinvestigated in medical personnel compared with patients.

Comparison of absolute benefits of anticancer therapies determined by snapshot and area methods

BACKGROUND

Reporting of relative risk reduction as the measure of treatment effect in randomized clinical trials (RCTs) may be difficult to understand. Here, we compare two methods for assessing absolute benefits of anticancer therapies.

MATERIALS AND METHODS

We searched PubMed for RCTs comparing therapies for breast and colorectal cancers published 1975-2009 (adjuvant setting) and 2000-2010 (metastatic setting). Eligible trials reported statistically significant differences. Kaplan-Meier curves were assessed for absolute differences in time-to-event end points at a single point (snapshot method) and as the area between curves (area method). Pooled absolute benefits determined by both methods were compared by the Pitman-Morgan test.

RESULTS

Eighty-three and 39 paired curves were assessed in the adjuvant and metastatic settings, respectively. In trials of adjuvant therapy, absolute benefits were larger and more variable when assessed at different time points by the snapshot compared with the area method (median and ranges for 60-month difference in overall survival: 7.6% [2.5%-28.4%] and 4.5% [1.8%-13.6%]; P = 0.002, respectively). For metastatic disease, both methods were within 0.5 month of each other in 62% of trials.

CONCLUSIONS

The area method provides an alternative measure of absolute treatment effect, which uses all of the available data and is less dependent on the shape of survival curves.

Using alternative statistical formats for presenting risks and risk reductions

BACKGROUND

The success of evidence-based practice depends on the clear and effective communication of statistical information.

OBJECTIVES

To evaluate the effects of using alternative statistical presentations of the same risks and risk reductions on understanding, perception, persuasiveness and behaviour of health professionals, policy makers, and consumers.

SEARCH STRATEGY

We searched Ovid MEDLINE (1966 to October 2007), EMBASE (1980 to October 2007), PsycLIT (1887 to October 2007), and the Cochrane Central Register of Controlled Trials (The Cochrane Library, 2007, Issue 3). We reviewed the reference lists of relevant articles, and contacted experts in the field.

SELECTION CRITERIA

We included randomized and non-randomized controlled parallel and cross-over studies. We focused on four comparisons: a comparison of statistical presentations of a risk (eg frequencies versus probabilities) and three comparisons of statistical presentation of risk reduction: relative risk reduction (RRR) versus absolute risk reduction (ARR), RRR versus number needed to treat (NNT), and ARR versus NNT.

DATA COLLECTION AND ANALYSIS

Two authors independently selected studies for inclusion, extracted data, and assessed risk of bias. We contacted investigators to obtain missing information. We graded the quality of evidence for each outcome using the GRADE approach. We standardized the outcome effects using adjusted standardized mean difference (SMD).

MAIN RESULTS

We included 35 studies reporting 83 comparisons. None of the studies involved policy makers. Participants (health professionals and consumers) understood natural frequencies better than probabilities (SMD 0.69 (95% confidence interval (CI) 0.45 to 0.93)). Compared with ARR, RRR had little or no difference in understanding (SMD 0.02 (95% CI -0.39 to 0.43)) but was perceived to be larger (SMD 0.41 (95% CI 0.03 to 0.79)) and more persuasive (SMD 0.66 (95% CI 0.51 to 0.81)). Compared with NNT, RRR was better understood (SMD 0.73 (95% CI 0.43 to 1.04)), was perceived to be larger (SMD 1.15 (95% CI 0.80 to 1.50)) and was more persuasive (SMD 0.65 (95% CI 0.51 to 0.80)). Compared with NNT, ARR was better understood (SMD 0.42 (95% CI 0.12 to 0.71)), was perceived to be larger (SMD 0.79 (95% CI 0.43 to 1.15)).There was little or no difference for persuasiveness (SMD 0.05 (95% CI -0.04 to 0.15)). The sensitivity analyses including only high quality comparisons showed consistent results for persuasiveness for all three comparisons. Overall there were no differences between health professionals and consumers. The overall quality of evidence was rated down to moderate because of the use of surrogate outcomes and/or heterogeneity. None of the comparisons assessed behaviourbehaviour.

AUTHORS' CONCLUSIONS

Natural frequencies are probably better understood than probabilities. Relative risk reduction (RRR), compared with absolute risk reduction (ARR) and number needed to treat (NNT), may be perceived to be larger and is more likely to be persuasive. However, it is uncertain whether presenting RRR is likely to help people make decisions most consistent with their own values and, in fact, it could lead to misinterpretation. More research is needed to further explore this question.

Abstracts in high profile journals often fail to report harm

BACKGROUND

To describe how frequently harm is reported in the abstract of high impact factor medical journals.

METHODS

DESIGN AND POPULATION

We carried out a blinded structured review of a random sample of 363 Randomised Controlled Trials (RCTs) carried out on human beings, and published in high impact factor medical journals in 2003. Main endpoint: 1) Proportion of articles reporting harm in the abstract; and 2) Proportion of articles that reported harm in the abstract when harm was reported in the main body of the article.

ANALYSIS

Corrected Prevalence Ratio (cPR) and its exact confidence interval were calculated. Non-conditional logistic regression was used.

RESULTS

363 articles and 407 possible comparisons were studied. Overall, harm was reported in 135 abstracts [37.2% (CI95%:32.2 to 42.4)]. Harm was reported in the main text of 243 articles [66.9% (CI95%: 61.8 to 71.8)] and was statistically significant in 54 articles [14.9% (CI95%: 11.4 to 19.0)]. Among the 243 articles that mentioned harm in the text, 130 articles [53.5% (CI95% 47.0 to 59.9)] reported harm in the abstract; a figure that rose to 75.9% (CI95%: 62.4 to 86.5) when the harm reported in the text was statistically significant. Harm in the abstract was more likely to be reported when statistically significant harm was reported in the main body of the article [cPR = 1.70 (CI95% 1.47 to 1.92)] and when drug companies (not public institutions) funded the RCTs [cPR = 1.29 (CI95% 1.03 to 1.67)].

CONCLUSION

Abstracts published in high impact factor medical journals underreport harm, even when harm is reported in the main body of the article.

What do we know about communicating risk? A brief review and suggestion for contextualising serious, but rare, risk, and the example of cox-2 selective and non-selective NSAIDs

BACKGROUND

Communicating risk is difficult. Although different methods have been proposed - using numbers, words, pictures or combinations - none has been extensively tested. We used electronic and bibliographic searches to review evidence concerning risk perception and presentation. People tend to underestimate common risk and overestimate rare risk; they respond to risks primarily on the basis of emotion rather than facts, seem to be risk averse when faced with medical interventions, and want information on even the rarest of adverse events.

METHODS

We identified observational studies (primarily in the form of meta-analyses) with information on individual non-steroidal anti-inflammatory drug (NSAID) or selective cyclooxygenase-2 inhibitor (coxib) use and relative risk of gastrointestinal bleed or cardiovascular event, the background rate of events in the absence of NSAID or coxib, and the likelihood of death from an event. Using this information we present the outcome of additional risk of death from gastrointestinal bleed and cardiovascular event for individual NSAIDs and coxibs alongside information about death from other causes in a series of perspective scales.

RESULTS

The literature on communicating risk to patients is limited. There are problems with literacy, numeracy and the human tendency to overestimate rare risk and underestimate common risk. There is inconsistency in how people translate between numbers and words. We present a method of communicating information about serious risks using the common outcome of death, using pictures, numbers and words, and contextualising the information. The use of this method for gastrointestinal and cardiovascular harm with NSAIDs and coxibs shows differences between individual NSAIDs and coxibs.

CONCLUSION

Although contextualised risk information can be provided on two possible adverse events, many other possible adverse events with potential serious consequences were omitted. Patients and professionals want much information about risks of medical interventions but we do not know how best to meet expectations. The impact of contextualised information remains to be tested.

Failure to adopt beneficial therapies caused by bias in medical evidence evaluation

BACKGROUND

Although it is known that many evidence-based therapies are underutilized, the causes of the research-practice gap are not well understood. The authors sought to determine if there is a bias in the evaluation of new evidence that leads to low rates of adoption of beneficial therapies compared to abandonment of harmful ones.

METHODS

Two case vignettes describing hypothetical clinical trials were administered to 2 independent samples of pulmonary and critical care practitioners. Each vignette was presented in 2 different ways; in one version, the results of the hypothetical trial showed that a treatment was harmful, and in the other version, the same treatment was shown to be beneficial. Prospective respondents from each sample were randomized to receive 1 version of each vignette (intersubject design). The main outcome was respondent's willingness to apply the results of the hypothetical trial to patient care.

RESULTS

There were 174 participants for trial 1 and 138 participants for trial 2 (enrollment rates of 44.2% and 41.8%, respectively). For trial 1, respondents were 2.3 times less likely to change clinical practice based on results indicating benefit as opposed to harm (33.3% v. 76.5%; P < 0.0001). Similarly, for trial 2, respondents were 2.57 times less likely to change practice when trial results showed that early use was beneficial as opposed to showing that early use was harmful (37.1% v. 95.3%; P < 0.0001).

CONCLUSIONS

When evaluating clinical trials, physicians demonstrate less willingness to adopt beneficial therapies than to abandon harmful ones. This difference may contribute to the research-practice gap.

Expressing medicine side effects: assessing the effectiveness of absolute risk, relative risk, and number needed to harm, and the provision of baseline risk information

OBJECTIVE

To assess the effectiveness of absolute risk, relative risk, and number needed to harm formats for medicine side effects, with and without the provision of baseline risk information.

METHODS

A two factor, risk increase format (relative, absolute and NNH)xbaseline (present/absent) between participants design was used. A sample of 268 women was given a scenario about increase in side effect risk with third generation oral contraceptives, and were required to answer written questions to assess their understanding, satisfaction, and likelihood of continuing to take the drug.

RESULTS

Provision of baseline information significantly improved risk estimates and increased satisfaction, although the estimates were still considerably higher than the actual risk. No differences between presentation formats were observed when baseline information was presented. Without baseline information, absolute risk led to the most accurate performance.

CONCLUSION

The findings support the importance of informing people about baseline level of risk when describing risk increases. In contrast, they offer no support for using number needed to harm.

PRACTICE IMPLICATIONS

Health professionals should provide baseline risk information when presenting information about risk increases or decreases. More research is needed before numbers needed to harm (or treat) should be given to members of the general populations.

Number needed to treat (or harm)

The effect of a treatment versus controls may be expressed in relative or absolute terms. For rational decision-making, absolute measures are more meaningful. The number needed to treat, the reciprocal of the absolute risk reduction, is a powerful estimate of the effect of a treatment. It is particularly useful because it takes into account the underlying risk (what would happen without the intervention?). The number needed to treat tells us not only whether a treatment works but how well it works. Thus, it informs health care professionals about the effort needed to achieve a particular outcome. A number needed to treat should be accompanied by information about the experimental intervention, the control intervention against which the experimental intervention has been tested, the length of the observation period, the underlying risk of the study population, and an exact definition of the endpoint. A 95% confidence interval around the point estimate should be calculated. An isolated number needed to treat is rarely appropriate to summarize the usefulness of an intervention; multiple numbers needed to treat for benefit and harm are more helpful. Absolute risk reduction and number needed to treat should become standard summary estimates in randomized controlled trials.