Decision making heuristics and the elicitation of preferences: being fast and frugal about the future

Comparison of Preferences and Data Quality between Discrete Choice Experiments Conducted in Online and Face-to-Face Respondents

INTRODUCTION

Discrete choice experiments (DCE) are increasingly being conducted using online panels. However, the comparability of such DCE-based preferences to traditional modes of data collection (e.g., in-person) is not well established. In this study, supervised, face-to-face DCE was compared with its unsupervised, online facsimile on face validity, respondent behavior, and modeled preferences.

METHODS

Data from face-to-face and online EQ-5D-5L health state valuation studies were compared, in which each used the same experimental design and quota sampling procedure. Respondents completed 7 binary DCE tasks comparing 2 EQ-5D-5L health states presented side by side (health states A and B). Data face validity was assessed by comparing preference patterns as a function of the severity difference between 2 health states within a task. The prevalence of potentially suspicious choice patterns (i.e., all As, all Bs, and alternating As/Bs) was compared between studies. Preference data were modeled using multinomial logit regression and compared based on dimensional contribution to overall scale and importance ranking of dimension-levels.

RESULTS

One thousand five Online respondents and 1,099 face-to-face screened (F2FS) respondents were included in the main comparison of DCE tasks. Online respondents reported more problems on all EQ-5D dimensions except for Mobility. The face validity of the data was similar between comparators. Online respondents had a greater prevalence of potentially suspicious DCE choice patterns ([Online]: 5.3% [F2FS] 2.9%, P = 0.005). When modeled, the relative contribution of each EQ-5D dimension differed between modes of administration. Online respondents weighed Mobility more importantly and Anxiety/Depression less importantly.

DISCUSSION

Although assessments of face validity were similar between Online and F2FS, modeled preferences differed. Future analyses are needed to clarify whether differences are attributable to preference or data quality variation between modes of data collection.

Informing a target product profile for rapid tests to identify HBV-infected pregnant women with high viral loads: a discrete choice experiment with African healthcare workers

BACKGROUND

Elimination of mother-to-child transmission of hepatitis B virus (HBV) requires infant immunoprophylaxis and antiviral prophylaxis for pregnant women with high viral loads. Since real-time polymerase chain reaction (RT-PCR), a gold standard for assessing antiviral eligibility, is neither accessible nor affordable for women living in low-income and middle-income countries (LMICs), rapid diagnostic tests (RDTs) detecting alternative HBV markers may be needed. To inform future development of the target product profile (TPP) for RDTs to identify highly viremic women, we used a discrete choice experiment (DCE) and elicited preference and trade-off of healthcare workers (HCW) in Africa between the following four attributes of fictional RDTs: price, time-to-result, diagnostic sensitivity, and specificity.

METHODS

Through an online questionnaire survey, we asked participants to indicate their preferred test from a set of two RDTs in seven choice tasks with varying levels of the four attributes. We used mixed multinomial logit models to quantify the utility gain or loss generated by each attribute. We attempted to define minimal and optimal criteria for test attributes that can satisfy ≥ 70% and ≥ 90% of HCWs, respectively, as an alternative to RT-PCR.

RESULTS

A total of 555 HCWs from 41 African countries participated. Increases in sensitivity and specificity generated significant utility and increases in cost and time-to-result generated significant disutility. The size of the coefficients for the highest attribute levels relative to the reference levels were in the following order: sensitivity (β = 3.749), cost (β = -2.550), specificity (β = 1.134), and time-to-result (β = -0.284). Doctors cared most about test sensitivity, while public health practitioners cared about cost and midwives about time-to-result. For an RDT with 95% specificity, costing 1 US$, and yielding results in 20 min, the minimally acceptable test sensitivity would be 82.5% and the optimally acceptable sensitivity would be 87.5%.

CONCLUSIONS

African HCWs would prefer an RDT with the following order of priority: higher sensitivity, lower cost, higher specificity, and shorter time-to-result. The development and optimization of RDTs that can meet the criteria are urgently needed to scale up the prevention of HBV mother-to-child transmission in LMICs.

Preferences for COVID-19 epidemic control measures among French adults: a discrete choice experiment

In this stated preferences study, we describe for the first time French citizens' preferences for various epidemic control measures, to inform longer-term strategies and future epidemics. We used a discrete choice experiment in a representative sample of 908 adults in November 2020 (before vaccination was available) to quantify the trade-off they were willing to make between restrictions on the social, cultural, and economic life, school closing, targeted lockdown of high-incidence areas, constraints to directly protect vulnerable persons (e.g., self-isolation), and measures to overcome the risk of hospital overload. The estimation of mixed logit models with correlated random effects shows that some trade-offs exist to avoid overload of hospitals and intensive care units, at the expense of stricter control measures with the potential to reduce individuals' welfare. The willingness to accept restrictions was shared to a large extent across subgroups according to age, gender, education, vulnerability to the COVID-19 epidemic, and other socio-demographic or economic variables. However, individuals who felt at greater risk from COVID-19, and individuals expressing high confidence in the governmental management of the health and economic crisis, more easily accepted all these restrictions. Finally, we compared the welfare impact of alternative strategies combining different epidemic control measures. Our results suggest that policies close to a targeted lockdown or with medically prescribed self-isolation were those satisfying the largest share of the population and achieving high gain in average welfare, while average welfare was maximized by the combination of all highly restrictive measures. This illustrates the difficulty in making preference-based decisions on restrictions.

Using Eye-Tracking Technology with Older People in Memory Clinics to Investigate the Impact of Mild Cognitive Impairment on Choices for EQ-5D-5L Health States Preferences

BACKGROUND

Population ageing is a phenomenon taking place in almost every global region. Current estimates indicate that 10-20% of older people in developed countries have mild cognitive impairment (MCI), with these percentages predicted to rise markedly by 2050.

OBJECTIVE

Our objective was to apply eye-tracking technology to investigate the information processes adopted by older people with and without MCI in determining preferences for health states in the five-level EuroQol-5 Dimensions (EQ-5D-5L) instrument.

METHODS

Older people (aged ≥ 65 years; including both patients and family carers) attending outpatient memory clinics in Southern Adelaide between July 2017 and June 2018, competent to read and converse in English and with a Mini-Mental State Examination (MMSE) cognition score of ≥ 19 were invited to participate. In total, 52 people met the inclusion criteria, of whom 20 (38%) provided informed consent and fully participated. Participants were categorised into two subgroups (each n = 10) for comparison based upon established MMSE cognition thresholds (19-23, lower MMSE indicative of MCI; ≥ 24, higher MMSE indicative of good cognition). A discrete-choice experiment (DCE) comprising a series of pairwise choices between alternative EQ-5D-5L health states of varying survival duration with differential levels of task complexity (approximated by the degree of attribute level overlap in each choice), was administered as a face-to-face interview with the participant wearing an eye-tracking device.

RESULTS

Attribute non-attendance (ANA) was higher for the lower MMSE subgroup than for the higher MMSE subgroup, although these differences were generally not statistically significant. ANA remained relatively low and consistent for participants with good cognition regardless of task complexity. In contrast, ANA increased notably in participants exhibiting MCI, increasing from 10% on average per participant in the lower MMSE subgroup with five attribute level overlap to 23% on average per participant in the lower MMSE subgroup with zero attribute level overlap.

CONCLUSIONS

This exploratory study provided important insights into the information processes adopted by older people with varying levels of cognitive functioning when choosing between alternative EQ-5D-5L health states of varying survival duration and specifically the relationships between cognitive capacity, task complexity and the extent of ANA. Recent advances in econometric modelling of health state valuation data have demonstrated the added value of capturing ANA information as this can be accounted for in the DCE data analysis, thereby improving the precision of model estimates. Eye-tracking technology can usefully inform the design, conduct and econometric modelling of DCEs, driving the inclusion of this rapidly growing population traditionally excluded from preference-elicitation studies of this nature.

For better or worse? Investigating the validity of best-worst discrete choice experiments in health

Discrete choice experiments (DCEs) are frequently used in health economics to measure preferences for nonmarket goods. Best-worst discrete choice experiment (BWDCE) has been proposed as a variant of the traditional "pick the best" approach. BWDCE, where participants choose the best and worst options, is argued to generate more precise preference estimates because of the additional information collected. However, the validity of the approach relies on two necessary conditions: (a) best and worst decisions provide similar information about preferences and (b) asking individuals to answer more than one choice question per task does not reduce data quality. Whether these conditions hold in empirical applications remains under researched. This is the first study to compare participants' choices across three experimental conditions: (a) BEST choices only, (b) WORST choices only, and (c) BEST and WORST choices (BWDCE). We find responses to worst choices are noisier. Implied preferences from the best only and worst only choices are qualitatively different, leading to different WTP values. Responses to BWDCE tasks have lower consistency, and respondents are more likely to use simplifying decision heuristics. We urge caution in using BWDCE as an alternative to the traditional "pick the best" DCE.

Measuring Patient Preferences: An Overview of Methods with a Focus on Discrete Choice Experiments

There is increasing recognition of the importance of patient preferences and methodologies to measure them. In this article, methods to quantify patient preferences are reviewed, with a focus on discrete choice experiments. In a discrete choice experiment, patients are asked to choose between 2 or more treatments. The results can be used to quantify the relative importance of treatment outcomes and/or other considerations relevant to medical decision making. Conducting and interpreting a discrete choice experiment requires multiple steps and an understanding of the potential biases that can arise, which we review in this article with examples in rheumatic diseases.

Response Patterns in Health State Valuation Using Endogenous Attribute Attendance and Latent Class Analysis

Not accounting for simplifying decision-making heuristics when modelling data from discrete choice experiments has been shown potentially to lead to biased inferences. This study considers two ways of exploring the presence of attribute non-attendance (that is, respondents considering only a subset of the attributes that define the choice options) in a health state valuation discrete choice experiment. The methods used include the latent class (LC) and endogenous attribute attendance (EAA) models, which both required adjustment to reflect the structure of the quality-adjusted life year (QALY) framework for valuing health outcomes. We find that explicit consideration of attendance patterns substantially improves model fit. The impact of allowing for non-attendance on the estimated QALY weights is dependent on the assumed source of non-attendance. If non-attendance is interpreted as a form of preference heterogeneity, then the inferences from the LC and EAA models are similar to those from standard models, while if respondents ignore attributes to simplify the choice task, the QALY weights differ from those using the standard approach. Because the cause of non-attendance is unknown in the absence of additional data, a policymaker may use the range of weights implied by the two approaches to conduct a sensitivity analysis.

An exploration of parents' preferences for foot care in juvenile idiopathic arthritis: a possible role for the discrete choice experiment

Background

An increased awareness of patients’ and parents’ care preferences regarding foot care is desirable from a clinical perspective as such information may be utilised to optimise care delivery. The aim of this study was to examine parents’ preferences for, and valuations of foot care and foot-related outcomes in juvenile idiopathic arthritis (JIA).

Methods

A discrete choice experiment (DCE) incorporating willingness-to-pay (WTP) questions was conducted by surveying 42 parents of children with JIA who were enrolled in a randomised-controlled trial of multidisciplinary foot care at a single UK paediatric rheumatology outpatients department. Attributes explored were: levels of pain; mobility; ability to perform activities of daily living (ADL); waiting time; referral route; and footwear. The DCE was administered at trial baseline. DCE data were analysed using a multinomial-logit-regression model to estimate preferences and relative importance of attributes of foot care. A stated-preference WTP question was presented to estimate parents’ monetary valuation of health and service improvements.

Results

Every attribute in the DCE was statistically significant (p < 0.01) except that of cost (p = 0.118), suggesting that all attributes, except cost, have an impact on parents’ preferences for foot care for their child. The magnitudes of the coefficients indicate that the strength of preference for each attribute was (in descending order): improved ability to perform ADL, reductions in foot pain, improved mobility, improved ability to wear desired footwear, multidisciplinary foot care route, and reduced waiting time. Parents’ estimated mean annual WTP for a multidisciplinary foot care service was £1,119.05.

Conclusions

In terms of foot care service provision for children with JIA, parents appear to prefer improvements in health outcomes over non-health outcomes and service process attributes. Cost was relatively less important than other attributes suggesting that it does not appear to impact on parents’ preferences.

Estimating the Danish populations' preferences for pharmacogenetic testing using a discrete choice experiment. The case of treating depression

OBJECTIVE

The objective is to estimate willingness-to-pay (WTP) for pharmacogenetic testing in the treatment of depression.

METHODS

In a web-based discrete choice questionnaire, four attributes were included: 1) number of changes in antidepressants before symptom relief; 2) time with dosage adjustments due to adverse side effects and/or lack of effects; 3) cost of pharmacogenetic testing; 4) probability of benefits from pharmacogenetic testing. Respondents were asked to choose between two scenarios; 1) pharmacogenetic testing; and 2) an opt-out option reflecting a scenario without pharmacogenetic testing. The indirect utility model was assumed to be multiplicative in probability of benefits and reduced time with dosage adjustments as well as reduced number of antidepressant changes.

RESULTS

Most coefficients had the expected signs and were statistically significant. WTP for avoidance of one change in antidepressant medication is 1571 Danish Krone (DKK), whereas WTP for reducing the period with dosage-adjustments by 1 month is DKK604. Both were statistically significantly different from zero.

CONCLUSION

If diagnosed with depression, peoples' WTP for pharmacogenetic testing appears to exceed its price as long as there is a reasonable probability for improvements in treatment (in the present case 10%). Utility is associated with outcomes only. Hence, other modes of provision of similar improvements in treatment may be valued equally highly. WTP estimates and the associated policy implications appear to be robust because they were unaffected by estimation model.

Why people refuse to make tradeoffs in person tradeoff elicitations: a matter of perspective?

OBJECTIVE

Person tradeoff (PTO) elicitations assess people's values for health states by asking them to compare the value of treatment programs. For example, people might be asked how many patients need to be cured of health condition X to equal the benefit of curing 100 people of condition Y. However, when faced with PTO elicitations, people frequently refuse to make quantifiable tradeoffs, exhibiting 2 kinds of refusals: 1) They say that 2 treatment programs have equal value, that curing 100 of X is just as good as curing 100 of Y, even if X is a less serious condition than Y, or 2) they say that the 2 programs are incomparable, that millions of people need to be cured of X to be as good as curing 100 of Y. The authors explore whether people would be more willing to make tradeoffs if the focus was changed from trading off groups of patients to choosing the best decision or evaluating treatment benefits.

DESIGN

. Two randomized trials used diverse samples (N=2400) via the Internet to test for the effect of perspective on refusal rates. The authors predicted that perspectives that removed people from decision-making roles would increase their willingness make tradeoffs.

RESULTS

Contrary to expectations, refusal rates increased when people were removed from decision-making roles. In fact, the more pressure put on people to make a decision, the less likely they were to refuse to make tradeoffs.

CONCLUSION

To reduce PTO refusals, it is best to adopt a decision-maker perspective.

Ordering effect and price sensitivity in discrete choice experiments: need we worry?

The objective of this paper is to analyse the impact that attribute ordering has on the relative importance of the price attribute. A discrete choice experiment was performed in order to elicit psoriasis patients' preferences for treatment. We tested for ordering effect with respect to the price attribute, and disclosed noticeable higher price sensitivity when the price attribute was placed at the end of the program description. Our results indicate that preferences are context dependent and that heuristics may be used in the choice process. Our result does not, however, suggest that ordering effect is a symptom of lexicographic ordering.

Deleting 'irrational' responses from discrete choice experiments: a case of investigating or imposing preferences?

Investigation of the 'rationality' of responses to discrete choice experiments (DCEs) has been a theme of research in health economics. Responses have been deleted from DCEs where they have been deemed by researchers to (a) be 'irrational', defined by such studies as failing tests for non-satiation, or (b) represent lexicographic preferences. This paper outlines a number of reasons why deleting responses from DCEs may be inappropriate after first reviewing the theory underpinning rationality, highlighting that the importance placed on rationality depends on the approach to consumer theory to which one ascribes. The aim of this paper is not to suggest that all preferences elicited via DCEs are rational. Instead, it is to suggest a number of reasons why it may not be the case that all preferences labelled as 'irrational' are indeed so. Hence, deleting responses may result in the removal of valid preferences; induce sample selection bias; and reduce the statistical efficiency and power of the estimated choice models. Further, evidence suggests random utility theory may be able to cope with such preferences. Finally, we discuss a number of implications for the design, implementation and interpretation of DCEs and recommend caution regarding the deletion of preferences from stated preference experiments.

The reasons of general practitioners for promoting colorectal cancer mass screening in France

Implementation of colorectal cancer (CRC) mass screening is a public health priority. Obtaining the involvement of general practitioners (GPs) is crucial for ensuring participation by the general public. This study elicited GPs' reasons for promoting CRC screening. The method used was discrete choice modeling. Questionnaires with a set of pairs of hypothetical screening scenarios were sent to 700 GPs. Factors influencing GPs' screening practice were found to be: the effectiveness of the screening program, the proportion of false negatives and false positives, and their remuneration. Hence improving screening test sensitivity and setting up specific remuneration for conducting screening would help to increase GPs' participation in CRC screening in France.

Using rank data to estimate health state utility models

In this paper we report the estimation of conditional logistic regression models for the Health Utilities Index Mark 2 and the SF-6D, using ordinal preference data. The results are compared to the conventional regression models estimated from standard gamble data, and to the observed mean standard gamble health state valuations. For both the HUI2 and the SF-6D, the models estimated using ordinal data are broadly comparable to the models estimated on standard gamble data and the predictive performance of these models is close to that of the standard gamble models. Our research indicates that ordinal data have the potential to provide useful insights into community health state preferences. However, important questions remain.

'Threats' to and hopes for estimating benefits

In a recent paper in this journal, Andrew Lloyd reviewed some potential threats to the estimation of health care benefits in monetary terms. Particular emphasis was placed on the extent to which the use of non-compensatory heuristics may distort the results. Although it is useful to be reminded of these problems, Lloyd's paper does not do justice to the attention such issues have already received, and continue to receive, within Health Economics. The aim of this paper is twofold. Firstly, it seeks to provide a more balanced and comprehensive view of the evidence by considering some of the methodological work that health economists have conducted in many of the areas pointed out by Lloyd. Secondly, and more importantly, it suggests ways to combine the economic and psychological views of human decision-making, providing a much more positive perspective to all those researchers out there who recognize the necessity of using stated preference methods to inform health policy.

Discrete-choice experiment to measure patient preferences for the surgical management of colorectal cancer

Background

Establishing trust between a patient and his or her surgeon is of paramount importance. The aim of this study was to assess the relative importance of the ‘attributes of trust’ between surgeon and patient with colorectal cancer.

Methods

A discrete-choice questionnaire was conducted with 60 men and 43 women who had completed primary treatment for colorectal cancer in two teaching hospitals in Sydney, Australia.

Results

Forty-seven of the 103 patients based their choice of surgical management on a single attribute and the remainder were willing to trade between different attributes. In order of importance, patients based their choice of surgical management on specialty training (β coefficient = 0·83), surgeon's communication (β = 0·82), type of hospital (β = 0·72) and who decides treatment (β = 0·01). Patients who were vigilant in their decision-making style and those who did not have tertiary education were more likely to change their preferences in the repeat interview.

Conclusion

Clinicians may have a better chance of meeting a patient's expectations about the process of care if they assess the patient's desire for knowledge and give those who do not have tertiary education more time to assimilate information about their treatment.

Discrete choice experiments in health economics. For better or for worse?

One method that is increasingly being used in health economics to elicit stated preferences concerning health matters is the discrete choice experiment (DCE). This editorial explores four sets of issues facing researchers who wish to employ DCE techniques: (a) normative issues about how data from DCE studies might be used to inform policy, (b) psychological issues concerning the meaningfulness of the data generated, (d) technical issues relating to how the data are generated and (d) issues relating to the generalisability of the data from DCE studies. Given current uncertainties surrounding these issues, it is our view that more caution and greater circumspection towards DCE is appropriate at this stage.