A systematic review of measurement uncertainty visualizations in the context of standardized assessments

This systematic review summarized findings of 29 studies evaluating visual presentation formats appropriate for communicating measurement uncertainty associated with standardized clinical assessment instruments. Studies were identified through systematic searches of multiple databases (Medline, Embase, PsycInfo, ERIC, Scopus, and Web of Science). Strikingly, we found no studies which were conducted using samples of clinicians and included clinical decision-making scenarios. Included studies did however find that providing participants with information about measurement uncertainty may increase awareness of uncertainty and promote more optimal decision making. Formats which visualize the shape of the underlying probability distribution were found to promote more accurate probability estimation and appropriate interpretations of the underlying probability distribution shape. However, participants in the included studies did not seem to benefit from the additional information provided by such plots during decision-making tasks. Further explorations into how presentations of measurement uncertainty impact clinical decision making are needed to examine whether findings of the included studies generalize to clinician populations. This review provides an important overview of pitfalls associated with formats commonly used to communicate measurement uncertainty in clinical assessment instruments, and a potential starting point for further explorations into promising alternatives. Finally, our review offers specific recommendations on how remaining research questions might be addressed.

Physical function and severe side effects matter most to patients with RA (< 5 years): a discrete choice experiment assessing preferences for personalized RA treatment

AIM

Early assessment of patient preferences has the potential to support shared decisions in personalized precision medicine for patients with rheumatoid arthritis (RA). The aim of this study was to assess treatment preferences of patients with RA (< 5 years) with previous experience of inadequate response to first-line monotherapy.

METHOD

Patients were recruited (March-June 2021) via four clinics in Sweden. Potential respondents (N = 933) received an invitation to answer a digital survey. The survey included an introductory part, a discrete choice experiment (DCE) and demographic questions. Each respondent answered 11 hypothetical choice questions as part of the DCE. Patient preferences and preference heterogeneity were estimated using random parameter logit models and latent class analysis models.

RESULTS

Patients (n = 182) assessed the most important treatment attributes out of physical functional capacity, psychosocial functional capacity, frequency of mild side effects and likelihood of severe side effects. In general, patients preferred a greater increase in functional capacity and decreased side effects. However, a substantial preference heterogeneity was identified with two underlying preference patterns. The most important attribute in the first pattern was the 'likelihood of getting a severe side effect'. Physical functional capacity was the most important attribute in the second pattern.

CONCLUSION

Respondents focused their decision-making mainly on increasing their physical functional capacity or decreasing the likelihood of getting a severe side effect. These results are highly relevant from a clinical perspective to strengthen communication in shared decision making by assessing patients' individual preferences for benefits and risks in treatment discussions.

Understanding what patients and physicians need to improve their decision-making about antenatal corticosteroids in late preterm gestation: a qualitative framework analysis

BACKGROUND

It is unclear whether the benefits of administration of antenatal corticosteroids in late preterm gestation outweigh its harms. We sought to understand whether patients and physicians need increased support to decide whether to administer antenatal corticosteroids in late preterm gestation, and their informational needs and preferences for decision-making roles related to this intervention; we also wanted to know if creation of a decision-support tool would be useful.

METHODS

We conducted individual, semistructured interviews with pregnant people, obstetricians and pediatricians in Vancouver, Canada, in 2019. Using a qualitative framework analysis method, we coded, charted and interpreted interview transcripts into categories that formed an analytical framework.

RESULTS

We included 20 pregnant participants, 10 obstetricians and 10 pediatricians. We organized codes into the following categories: informational needs to decide whether to administer antenatal corticosteroids; preferences for decision-making roles regarding this treatment; the need for support to make this treatment decision; and the preferred format and content of a decision-support tool. Pregnant participants wanted to be involved in decision-making about antenatal corticosteroids in late preterm gestation. They wanted information on the medication, respiratory distress, hypoglycemia, parent-neonate bonding and long-term neurodevelopment. There was variation in physician counselling practices, and in how patients and physicians perceived the balance of treatment harms and benefits. Responses suggested a decision-support tool may be useful. Participants desired clear descriptions of risk magnitude and uncertainty.

INTERPRETATION

Pregnant people and physicians would likely benefit from increased support to consider the harms and benefits of antenatal corticosteroids in late preterm gestation. Creation of a decision-support tool may be useful.

Investigating the presentation of uncertainty in an icon array: A randomized trial

BACKGROUND

Clinicians are often advised to use pictographs to communicate risk, but whether they offer benefits when communicating risk imprecision (e.g., 65%-79%) is unknown.

PURPOSE

To test whether any of three approaches to visualizing imprecision would more effectively communicate breast and ovarian cancer risk for BRCA1 pathogenic variant carriers.

METHODS

1,300 UK residents were presented with a genetic report with information about BRCA1-related risks, with random assignment to one of four formats: no visualization (text alone), or a pictograph using shaded icons, a gradient, or arrows marking range endpoints. We also tested pictographs in two layouts. Analysis of variance (ANOVA) and regression was employed.

RESULTS

There was no effect of format. Participants shown pictographs vs. text alone had better uptake of breast cancer risk messages (p < .05, η ² = 0.003). Pictographs facilitated memory for the specific amount of risk (p < 0.001, η ² = 0.019), as did the tabular layout. Individuals not having completed upper secondary education may benefit most.

CONCLUSIONS

We found weak evidence in favor of using simple pictographs with ranges to communicate BRCA risk (versus text alone), and of the tabular layout.

INNOVATION

Testing different ways of communicating imprecision within pictographs is a novel and promising line of research.

Fear of the Unknown: Can We Help Individuals With a Systemic Autoimmune Rheumatic Disease Deal With Uncertainty?

Unfortunately, not much is certain in systemic autoimmune rheumatic diseases (SARDs). People with a SARD such as systemic lupus erythematosus (SLE) and systemic sclerosis (SSc) are dealing with a chronic, inflammatory, and often unpredictable autoimmune condition that might cause them to experience illness-related uncertainty.^1,2.

Systematic review of quantitative preference studies of treatments for rheumatoid arthritis among patients and at-risk populations

Treatments used for rheumatoid arthritis (RA) are under investigation for their efficacy to prevent RA in at risk groups. It is therefore important to understand treatment preferences of those at risk. We systematically reviewed quantitative preference studies of drugs to treat, or prevent RA, to inform the design of further studies and trials of RA prevention. Stated preference studies for RA treatment or prevention were identified through a search of five databases. Study characteristics and results were extracted, and the relative importance of different types of treatment attributes was compared across populations. Twenty three studies were included 20 of RA treatments (18 of patients; 2 of the general public) and 3 prevention studies with first-degree relatives (FDRs). Benefits, risks, administration method and cost (when included) were important determinants of treatment choice. A benefit was more important than a risk attribute in half of the studies of RA treatment that included a benefit attribute and 2/3 studies of RA prevention. There was variability in the relative importance of attributes across the few prevention studies. In studies with non-patient participants, attributes describing confidence in treatment effectiveness/safety were more important determinants of choice than in studies with patients. Most preference studies relating to RA are of treatments for established RA. Few studies examine preferences for treatments to prevent RA. Given intense research focus on RA prevention, additional preference studies in this context are needed. Variation in treatment preferences across different populations is not well understood and direct comparisons are needed.

Views of Israeli healthcare professionals regarding communication of genetic variants of uncertain significance to patients

While genomic medicine is becoming an important part of patient care with an ever‐increasing diagnostic yield, communicating variants of uncertain clinical significance (VUSs) remains a major challenge. We draw on qualitative analysis of semi‐structured interviews conducted in 2020 with 20 Israeli healthcare professionals and stakeholders involved in communicating the results of genome‐wide sequencing to patients. Respondents described four main strategies of communicating VUSs to patients: preparing the patient pre‐test for uncertainty; adapting the level of detail to the patient's needs; upgrading versus downgrading the VUS; and following up on the possible reclassification of VUSs. These strategies were expressed differently by physicians and genetic counselors, varying according to their specialty and perception of the patient's situation. We discuss the strategic management and communication of uncertain genomic test results with patients in the context of meeting patients' expectations and working toward genetic causality through genomic narration and designation.

Communicating Uncertainties About the Effects of Medical Interventions Using Different Display Formats

Communicating uncertainties in scientific evidence is important to accurately reflect scientific knowledge , increase public understanding of uncertainty, and to signal transparency and honesty in reporting. While techniques have been developed to facilitate the communication of uncertainty, many have not been empirically tested, compared for communicating different types of uncertainty, or their effects on different cognitive, trust, and behavioral outcomes have not been evaluated. The present study examined how a point estimate, imprecise estimate, conflicting estimates, or a statement about the lack of evidence about treatment effects, influenced participant's responses to communications about medical evidence. For each type of uncertainty, we adapted three display formats to communicate the information: tables, bar graphs, and icon arrays. We compared participant's best estimates of treatment effects, as well as effects on recall, subjective evaluations (understandability and usefuleness), certainty perceptions, perceptions of trustworthiness of the information, and behavioral intentions. We did not find any detrimental effects from communicating imprecision or conflicting estimates relative to a point estimate across any outcome. Furthermore, there were more favorable responses to communicating imprecision or conflicting estimates relative to lack of evidence, where participants estimated the treatment would improve outcomes by 30-50% relative to a placebo. There were no differences across display formats, suggesting that, if well-designed, it may not matter which format is used. Future research on specific display formats or uncertainty types and with larger sample sizes would be needed to detect small effects. Implications for the communication of uncertainty are discussed.

Functional capacity vs side effects: treatment attributes to consider when individualising treatment for patients with rheumatoid arthritis

Introduction

Individualisation of rheumatoid arthritis (RA) treatment needs to take account of individual patients’ preferences to increase patient-centeredness in treatment decisions. The aim of this study was to identify patient-relevant treatment attributes to consider when individualising treatment for patients with RA.

Method

Patients with RA in Sweden were invited to rank the most important treatment attributes in an online survey (April to May 2020). Semi-structured interviews were conducted (October to November 2020) to further identify and frame potential attributes for shared decision-making. The interviews were audio-recorded, transcribed and analysed using thematic framework analysis. Patient research partners and rheumatologists supported the selection and framing of the treatment attributes across the assessment.

Results

The highest ranked attributes (N = 184) were improved functional capacity, reduced inflammation, reduced pain and fatigue and the risk of getting a severe side effect. The framework analysis revealed two overarching themes for further exploration: treatment goals and side effects. ‘Treatment goals’ emerged from functional capacity, revealing two dimensions: physical functional capacity and psychosocial functional capacity. ‘Side effects’ revealed that mild and severe side effects were the most important to discuss in shared decision-making.

Conclusions

Functional capacity (physical and psychosocial) and potential side effects (mild and severe) are important treatment attributes to consider when individualising RA treatment. Future research should assess how patients with RA weigh benefits and risks against each other, in order to increase patient-centeredness early on the treatment trajectory.

Current Best Practice for Presenting Probabilities in Patient Decision Aids: Fundamental Principles

BACKGROUND

Shared decision making requires evidence to be conveyed to the patient in a way they can easily understand and compare. Patient decision aids facilitate this process. This article reviews the current evidence for how to present numerical probabilities within patient decision aids.

METHODS

Following the 2013 review method, we assembled a group of 9 international experts on risk communication across Australia, Germany, the Netherlands, the United Kingdom, and the United States. We expanded the topics covered in the first review to reflect emerging areas of research. Groups of 2 to 3 authors reviewed the relevant literature based on their expertise and wrote each section before review by the full authorship team.

RESULTS

Of 10 topics identified, we present 5 fundamental issues in this article. Although some topics resulted in clear guidance (presenting the chance an event will occur, addressing numerical skills), other topics (context/evaluative labels, conveying uncertainty, risk over time) continue to have evolving knowledge bases. We recommend presenting numbers over a set time period with a clear denominator, using consistent formats between outcomes and interventions to enable unbiased comparisons, and interpreting the numbers for the reader to meet the needs of varying numeracy.

DISCUSSION

Understanding how different numerical formats can bias risk perception will help decision aid developers communicate risks in a balanced, comprehensible manner and avoid accidental "nudging" toward a particular option. Decisions between probability formats need to consider the available evidence and user skills. The review may be useful for other areas of science communication in which unbiased presentation of probabilities is important.

Basing Information on Comprehensive, Critically Appraised, and Up-to-Date Syntheses of the Scientific Evidence: An Update from the International Patient Decision Aid Standards

BACKGROUND

Patients and clinicians expect the information in patient decision aids to be based on the best available research evidence. The objectives of this International Patient Decision Aid Standards (IPDAS) review were to 1) check the currency of, and where needed, update evidence for the domain of "basing the information in decision aids on comprehensive, critically appraised, and up-to-date syntheses of the evidence"; 2) analyze the evidence characteristics of decision aids; and 3) propose updates to relevant IPDAS criteria.

METHODS

We searched MEDLINE and PubMed to inform updates of this domain's definitions, justifications, and components. We also searched 5 sources to identify all publicly available decision aids (N = 471). Two assessors independently extracted each aid's evidence characteristics.

RESULTS

Minor updates to the definitions and theoretical justifications of this IPDAS domain are provided and changes to relevant IPDAS criteria proposed. Nearly all aids (97%) provided a year of creation/update, but most (81%) did not report an explicit update or expiration policy. No scientific references were cited in 33% of aids. Of the 314 that cited at least 1 reference, 39% cited at least 1 guideline, 44% cited at least 1 systematic review, and 23% cited at least 1 randomized trial. In 35%, it was unclear what statement in the aid the citations referred to. Only 14% reported any of the processes used to find and decide on evidence inclusion. Only 14% reported the evidence quality. Many emerging issues and future research areas were identified.

CONCLUSIONS

Although many emerging issues need to be addressed, this IPDAS domain is validated and criteria refined. High-quality patient decision aids should be based on comprehensive and up-to-date syntheses of critically appraised evidence.

Conceptualising, evaluating and communicating uncertainty in forensic science: Identifying commonly used tools through an interdisciplinary configurative review

This study provides a set of tools for conceptualising, evaluating and communicating uncertainty in forensic science. Given that the concept of uncertainty is one that transcends disciplinary boundaries, an interdisciplinary configurative review was carried out incorporating the disciplines of medicine, environmental science and economics, in order to identify common themes which could have valuable applications to the discipline of forensic science. Critical Interpretive Synthesis was used to develop sub-synthetic and synthetic constructs which interpreted and synthesised the underlying evidence and codes. This study provides three toolkits, one each for conceptualisation, evaluation and communication. The study identified an underlying theme concerning the obstacles that would need to be overcome for the effective application of these toolkits and achieving effective conceptualisation, evaluation and communication of uncertainty in forensic science to lay-stakeholders. These toolkits offer a starting point for developing the conversation for achieving greater transparency in the communication of uncertainty. They also have the potential to offer stakeholders enhanced understanding of the nuances and limitations of forensic science evidence and enable more transparent evaluation and scrutiny of the reliability, relevance and probative value of forensic materials in a crime reconstruction.

Risk communication in a patient decision aid for radiotherapy in breast cancer: How to deal with uncertainty?

Background and aim

Patient decision aids for oncological treatment options, provide information on the effect on recurrence rates and/or survival benefit, and on side-effects and/or burden of different treatment options. However, often uncertainty exists around the probability estimates for recurrence/survival and side-effects which is too relevant to be ignored. Evidence is lacking on the best way to communicate these uncertainties. The aim of this study is to develop a method to incorporate uncertainties in a patient decision aid for breast cancer patients to support their decision on radiotherapy.

Methods

Firstly, qualitative interviews were held with patients and health care professionals. Secondly, in the development phase, thinking aloud sessions were organized with four patients and 12 health care professionals, individual and group-wise.

Results

Consensus was reached on a pictograph illustrating the whole range of uncertainty for local recurrence risks, in combination with textual explanation that a more exact personalized risk would be given by their own physician. The pictograph consisted of 100 female icons in a 10 x 10 array. Icons with a stepwise gradient color indicated the uncertainty margin. The prevalence and severity of possible side-effects were explained using verbal labels.

Conclusions

We developed a novel way of visualizing uncertainties in recurrence rates in a patient decision aid. The effect of this way of communicating risk uncertainty is currently being tested in the BRASA study (NCT03375801).

•

There exists uncertainty around local recurrence risks for breast cancer patients.

•

Little is known on how to communicate uncertainty to patients.

•

Patient decision aids can help communicating risks and uncertainty.

•

We developed pictographs to communicate numerical uncertainty in recurrence risks.

•

The effect of the pictographs is currently being tested in the BRASA study.

Communicating Uncertainty: a Narrative Review and Framework for Future Research

Discussing the uncertainty associated with a clinical decision is thought to be a critical element of shared decision-making. Yet, empirical evidence suggests that clinicians rarely communicate clinical uncertainty to patients, and indeed the culture within healthcare environments is often to equate uncertainty with ignorance or failure. Understanding the rationale for discussion of uncertainty along with the current evidence about approaches to communicating and managing uncertainty can advance shared decision-making as well as highlight gaps in evidence. With an increasing focus on personalized healthcare, and advances in genomics and new disease biomarkers, a more sophisticated understanding of how to communicate the limitations and errors that come from applying population-based, epidemiologic findings to predict individuals' futures is going to be essential. This article provides a narrative review of studies relating to the communication of uncertainty, highlighting current strategies together with challenges and barriers, and outlining a framework for future research.

Preferences of Patients and At-risk Individuals for Preventive Approaches to Rheumatoid Arthritis

Effective treatments for rheumatoid arthritis (RA) are available and can lead to remission for some patients, but most patients remain on potentially toxic and expensive medications in the long term. Interest is increasingly turning to the disease phases preceding the development of RA that represent opportunities for preventive interventions. At-risk target populations include individuals with genetic and environmental risk factors, those who have developed systemic autoimmunity, and those who have developed clinically suspect symptoms (eg, arthralgias without synovitis, or an early arthritis). Ongoing prospective studies will inform the development of increasingly accurate predictive tools to identify individuals at risk of developing RA. Furthermore, a range of preventive approaches has been suggested, including lifestyle modification (eg, smoking cessation) and pharmacologic interventions (eg, hydroxychloroquine, methotrexate, abatacept, rituximab) that are currently the subject of randomized controlled trials. As prediction and prevention of RA evolve, it is increasingly likely that individuals at risk (including asymptomatic individuals) may be faced with complex decisions about whether to accept assessment of their risk status or to take a preventive intervention associated with risk of serious adverse events and uncertain benefit. Acceptance of preventive medication in other contexts can be low. For example, <25% of women at high risk of breast cancer are willing to take preventive hormonal treatments. Actual uptake is lower still. Patients' beliefs and preferences predict treatment uptake and adherence. Before the dream of preventing RA can become reality, health care providers need to understand the perspectives of individuals in the target population and to identify barriers and facilitators for this approach. This commentary reviews what is currently known about the perspectives of patients and individuals at risk about predictive and preventive approaches for RA and identifies gaps to be addressed to inform the development of efficient preventive strategies.

Using Discrete Choice Experiments to Inform the Benefit-Risk Assessment of Medicines: Are We Ready Yet?

There is emerging interest in the use of discrete choice experiments as a means of quantifying the perceived balance between benefits and risks (quantitative benefit-risk assessment) of new healthcare interventions, such as medicines, under assessment by regulatory agencies. For stated preference data on benefit-risk assessment to be used in regulatory decision making, the methods to generate these data must be valid, reliable and capable of producing meaningful estimates understood by decision makers. Some reporting guidelines exist for discrete choice experiments, and for related methods such as conjoint analysis. However, existing guidelines focus on reporting standards, are general in focus and do not consider the requirements for using discrete choice experiments specifically for quantifying benefit-risk assessments in the context of regulatory decision making. This opinion piece outlines the current state of play in using discrete choice experiments for benefit-risk assessment and proposes key areas needing to be addressed to demonstrate that discrete choice experiments are an appropriate and valid stated preference elicitation method in this context. Methodological research is required to establish: how robust the results of discrete choice experiments are to formats and methods of risk communication; how information in the discrete choice experiment can be presented effectually to respondents; whose preferences should be elicited; the correct underlying utility function and analytical model; the impact of heterogeneity in preferences; and the generalisability of the results. We believe these methodological issues should be addressed, alongside developing a 'reference case', before agencies can safely and confidently use discrete choice experiments for quantitative benefit-risk assessment in the context of regulatory decision making for new medicines and healthcare products.

Valuing health for oneself versus one's child or elderly parent

The aim of this study was to determine if adults value health states or are willing to accept risk differently for themselves than for their children or elderly parents. Participants (701) were asked to rate four hypothetical health states for themselves using both the standard gamble and time trade-off methodologies. They then did the same assessments for a real or hypothetical child as well as an elderly parent. Participants were willing to take more risk or trade more years of life to avoid bilateral vision loss and mental impairment for themselves than they were for their children and elderly parents.

Preferences for 'New' Treatments Diminish in the Face of Ambiguity

New products usually offer advantages over existing products, but in health care, most new drugs are 'me-too', comparable in effectiveness and side effects to existing drugs, but with a more ambiguous evidence base around adverse effects. Despite this, new treatments drive increased health care spending, suggesting a preference for 'newness' in this setting. We explore (1) whether preferences for treatments labeled 'new' exist and (2) persist once the ambiguity in the evidence base reflecting newness is described. We use a Canadian general population sample (n = 2837) characterized by their innovativeness in adopting new products in normal markets. We found that innovators/early adopters (n = 173) had significant preferences for 'newer' treatments (B = 0.162, p = 0.038) irrespective of comparable benefits and side effects and all respondents had significant preferences for less ambiguity in benefit/side effect estimates. Notably, when 'newness' was combined with ambiguity, no significant preferences for new treatments were observed regardless of respondent innovativeness. We conclude that preferences for new products exist for some people in health care markets but disappear when the implication of ambiguity in the evidence base for new treatments is communicated. Physicians should avoid describing treatments as 'new' or be mindful to qualify the implications of 'new' treatments in terms of evidence ambiguity. Copyright © 2016 John Wiley & Sons, Ltd.

Attitude to health risk in the Canadian population: a cross-sectional survey

BACKGROUND

Risk is a ubiquitous part of health care. Understanding how people respond to risks is important for predicting how populations make health decisions. Our objective was to seek preliminary descriptive insights into the attitude to health risk in the Canadian population and factors associated with heterogeneity in risk attitude.

METHODS

We used a large market-research panel to survey (in English and French) a representative sample of the Canadian general population that reflected the age, sex and geography of the population. The survey included the Health-Risk Attitude Scale, which predicts how a person resolves risky health decisions related to treatment, prevention of disease and health-related behaviour. In addition, we assessed participants' numeracy and risk understanding, as well as income band and level of education. We summarized the responses, and we explored the independent associations between demographics, numeracy, risk understanding and risk attitude in multivariable models.

RESULTS

Of 6780 respondents, 4949 (73.0%) were averse to health risks; however, but there was considerable heterogeneity in the magnitude of risk aversion. We found significant gradients of risk averse attitudes with increasing age and being female (p < 0.001) using the multivariable model. French-speaking participants appeared to be more risk averse than those who were English-speaking (p < 0.001), as were individuals scoring higher on the Subjective Numeracy Scale (p < 0.001).

INTERPREATION

In general, Canadians were averse to health risks, but we found that a sizeable, identifiable group of risk takers exists. Heterogeneity in preferences for risk can explain variations in health care utilization in the context of patient-centred care. Understanding risk preference heterogeneity can help guide policy and assist in patient-physician decisions.