Who pays attention in stated-choice surveys?

Choice Consistency in Discrete Choice Experiments: Does Numeracy Skill Matter?

OBJECTIVES

This study investigated the relationship between numeracy skills (NS) and choice consistency in discrete choice experiments (DCEs).

METHODS

A DCE was conducted to explore patients' preferences for kidney transplantation in Italy. Patients completed the DCE and answered 3-item numeracy questions. A heteroskedastic multinomial logit model was used to investigate the effect of numeracy on choice consistency.

RESULTS

Higher NS were associated with greater choice consistency, increasing the scale to 1.63 (P < .001), 1.39 (P < .001), and 1.18 (P < .001) for patients answering 3 of 3, 2 of 3, and 1 of 3 questions correctly, respectively, compared with those with no correct answers. This corresponded to 63%, 39%, and 18% more consistent choices, respectively. Accounting for choice consistency resulted in varying willingness-to-wait (WTW) estimates for kidney transplant attributes. Patients with the lowest numeracy (0/3) were willing to wait approximately 42 months [95% CI: 29.37, 54.68] for standard infectious risk, compared with 33 months [95% CI: 28.48, 38.09] for 1 of 3, 28 months [95% CI: 25.13, 30.32] for 2 of 3, and 24 months [95% CI: 20.51, 27.25] for 3 of 3 correct answers. However, WTW differences for an additional year of graft survival and neoplastic risk were not statistically significant across numeracy levels. Supplementary analyses of 2 additional DCEs on COVID-19 vaccinations and rheumatoid arthritis, conducted online, supported these findings: higher NS were associated with more consistent choices across different disease contexts and survey formats.

CONCLUSIONS

The findings suggested that combining patients with varying NS could bias WTW estimates, highlighting the need to consider numeracy in DCE data analysis and interpretation.

Getting it right with discrete choice experiments: Are we hot or cold?

Discrete Choice Experiments (DCEs) are widely employed survey-based methods to assess preferences for healthcare services and products. While they offer an experimental way to represent health-related decisions, the stylized representation of scenarios in DCEs may overlook contextual factors that could influence decision-making. The aim of this paper was to evaluate the predictive validity of preferences elicited through a DCE in decisions likely influenced by a hot-cold empathy gap, and compare it to another commonly used method, a direct-elicitation question. We focused on preferences for pain-relief modalities, especially for an epidural during childbirth - a context where direct-elicitation questions have shown a preference for or intention to have a natural birth (representing the "cold" state), yet individuals often opt for an epidural during labor (representing the "hot" state). Leveraging a unique dataset collected from 248 individuals, we incorporated both the stated preferences collected through a survey administered upon hospital admission for childbirth and the actual pain-relief modality usage data documented in medical records. The DCE allowed for the evaluation of scenarios outside of those expected by respondents to simulate decision-making during childbirth. When we compared the predicted epidural use with the actual epidural use during labor, we observed a choice concordance of 71-60%, depending on the model specification. The concordance rate between the predicted and actual choices increased to 77-76% when accounting for the initial use of other ineffective modalities. In contrast, the direct-elicitation choices, relying solely on respondents' baseline expectations, yielded a lower concordance rate of 58% with actual epidural use. These findings highlight the flexibility of the DCE method in simulating complex decision contexts, including those involving hot-cold empathy gaps. The DCE proves valuable in assessing nuanced preferences, providing a more accurate representation of the decision-making processes in healthcare scenarios.

An Overview of Data Collection in Health Preference Research

This paper focuses on survey administration and data collection methods employed for stated-preference studies in health applications. First, it describes different types of survey administration methods, encompassing web-based surveys, face-to-face (in-person) surveys, and mail surveys. Second, the concept of sampling frames is introduced, clarifying distinctions between the target population and survey frame population. The discussion then extends to different types of sampling methods, such as probability and non-probability sampling, along with an evaluation of potential issues associated with different sampling methods within the context of health preference research. Third, the paper provides information about different recruitment methods, including web-surveys, leveraging patient groups, and in-clinic recruitment. Fourth, a crucial aspect addressed is the calculation of response rate, with insights into determining an adequate response rate and strategies to improve response rates in stated-preference surveys. Lastly, the paper concludes by discussing data management plans and suggesting insights for future research in this field. In summary, this paper examines the nuanced aspects of survey administration and data collection methods in stated-preference studies, offering valuable guidance for researchers and practitioners in the health domain.

Do people have differing motivations for participating in a stated-preference study? Results from a latent-class analysis

BACKGROUND

Researchers and policy makers have long suspected that people have differing, and potentially nefarious, motivations for participating in stated-preference studies such as discrete-choice experiments (DCE). While anecdotes and theories exist on why people participate in surveys, there is a paucity of evidence exploring variation in preferences for participating in stated-preference studies.

METHODS

We used a DCE to estimate preferences for participating in preference research among an online survey panel sample. Preferences for the characteristics of a study to be conducted at a local hospital were assessed across five attributes (validity, relevance, bias, burden, time and payment) and described across three levels using a starring system. A D-efficient experimental design was used to construct three blocks of 12 choice tasks with two profiles each. Respondents were also asked about factors that motivated their choices. Mixed logistic regression was used to analyze the aggregate sample and latent class analysis identified segments of respondents.

RESULTS

629 respondents completed the experiment. In aggregate "study validity" was most important. Latent class results identified two segments based on underlying motivations: a quality-focused segment (76%) who focused most on validity, relevance, and bias and a convenience-focused segment (24%) who focused most on reimbursement and time. Quality-focused respondents spent more time completing the survey (p < 0.001) and were more likely to identify data quality (p < 0.01) and societal well-being (p < 0.01) as motivations to participate.

CONCLUSIONS

This information can be used to better understand variability in motivations to participate in stated-preference surveys and the impact of motivations on response quality.

Text messaging and financial incentives to encourage weight loss in men with obesity: the Game of Stones feasibility RCT

Background

In 2016, 26% of UK men were estimated to be obese. Systematic reviews suggest that few men engage in formal weight loss interventions that support weight reduction and improve health.

Objective

To co-produce, with patient and public involvement, an acceptable and feasible randomised controlled trial design to test a men-only weight management intervention.

Design

This was a two-phase feasibility study. Phase 1 was the development of intervention components, study procedures and materials including a discrete choice experiment with survey questions. Phase 2 was an individually randomised three-arm feasibility trial over 12 months. Qualitative interviews were conducted at 3 and 12 months.

Setting

The setting was two sites in Scotland that had disadvantaged urban and rural areas and differed in employment levels and ethnic groups.

Participants

In phase 1, 1045 men with obesity were recruited by Ipsos MORI (London, UK; www.ipsos.com/ipsos-mori/en-uk) to represent the UK population. In phase 2, 105 men with obesity were recruited in the community or through general practice obesity registers. Qualitative interviews were conducted with 50 men at 3 months and with 33 men at 12 months.

Interventions

The trial arms were narrative short message service (SMS) for 12 months (SMS only), financial endowment incentive informed by loss aversion and linked to achievement of weight loss targets plus narrative SMS for 12 months (SMS + I), and waiting list control group for 12 months followed by 3 months of an alternative SMS style developed based on feedback from men who had received the narrative SMS (control).

Main outcome measures

The main outcome measures were acceptability and feasibility of recruitment, retention, engagement, intervention components and trial procedures. Outcomes were assessed by examining procedural, quantitative and qualitative data at 3, 6 and 12 months.

Results

The most acceptable incentive strategy, based on the discrete choice experiment results, was to verify weight loss of 5% at 3 months, verify weight loss of 10% at 6 months and maintain weight loss of 10% at 12 months. Overall, 105 men with obesity from across the socioeconomic spectrum were successfully recruited to target, 59% of whom lived in more disadvantaged areas. Retention at 12 months was acceptable (74%) and was higher among individuals from disadvantaged areas. Narrative SMS were acceptable to many men, with a minority reporting negative reactions. Incentives were acceptable but were not the primary motivation for behaviour change. Twelve men in the incentive arm (33%) secured at least some money and three (8%) secured the full amount. Both intervention arms lost some weight, with greater weight loss in the arm that received SMS and incentives. The alternative SMS based on men’s feedback received no strong negative reactions.

Limitations

Fewer participants from the SMS + I arm (64%) completed the study at 12 months than did those in the SMS-only (79%) and control (83%) arms. The reasons for this difference were complex.

Conclusions

The men-only weight management intervention consisting of narrative SMS and financial incentives was acceptable and feasible, meeting the progression criteria for a full trial. Tailoring of SMS may improve acceptability and retention.

Future work

Minor refinements to the intervention components based on the study findings will be made prior to testing in a multisite definitive randomised controlled trial.

Trial registration

ClinicalTrials.gov NCT03040518.

Funding

This project was funded by the National Institute for Health Research (NIHR) Public Health Research programme and will be published in full in Public Health Research; Vol. 8, No. 11. See the NIHR Journals Library website for further project information.

Preferences for centralised emergency medical services: discrete choice experiment

OBJECTIVES

It is desirable that public preferences are established and incorporated in emergency healthcare reforms. The aim of this study was to investigate preferences for local versus centralised provision of all emergency medical services (EMS) and explore what individuals think are important considerations for EMS delivery.

DESIGN

A discrete choice experiment was conducted. The attributes used in the choice scenarios were: travel time to the hospital, waiting time to be seen, length of stay in the hospital, risks of dying, readmission and opportunity for outpatient care after emergency treatment at a local hospital.

SETTING

North East England.

PARTICIPANTS

Participants were a randomly sampled general population, aged 16 years or above recruited from Healthwatch Northumberland network database of lay members and from clinical contact with Northumbria Healthcare National Health Service Foundation Trust via Patient Experience Team.

PRIMARY AND SECONDARY OUTCOME MEASURES

Analysis used logistic regression modelling techniques to determine the preference of each attribute. Marginal rates of substitution between attributes were estimated to understand the trade-offs individuals were willing to make.

RESULTS

Responses were obtained from 148 people (62 completed a web and 86 a postal version). Respondents preferred shorter travel time to hospital, shorter waiting time, fewer number of days in hospital, low risk of death, low risk of readmission and outpatient follow-up care in their local hospital. However, individuals were willing to trade off increased travel time and waiting time for high-quality centralised care. Individuals were willing to travel 9 min more for a 1-day reduction in length of stay in the hospital, 38 min for a 1% reduction in risk of death and 112 min for having outpatient follow-up care at their local hospital.

CONCLUSIONS

People value centralised EMS if it provides higher quality care and are willing to travel further and wait longer.

Development of a Discrete Choice Experiment (DCE) Questionnaire to Understand Veterans' Preferences for Tobacco Treatment in Primary Care

BACKGROUND

Providers often prescribe counseling and/or medications for tobacco cessation without considering patients' treatment preferences.

OBJECTIVE

The primary aims of this study are to describe (1) the development of a discrete choice experiment (DCE) questionnaire designed to identify the attributes and levels of tobacco treatment that are most important to veterans; and (2) the decision-making process in choosing between hypothetical tobacco treatments.

METHODS

We recruited current smokers who were already scheduled for a primary care appointment within a single Veterans Affairs (VA) healthcare system. Subjects were asked to rate the importance of selected treatment attributes and were interviewed during two rounds of pilot testing of initial DCE instruments. Key attributes and levels of the initial instruments were identified by targeted literature review; the instruments were iteratively revised after each round of pilot testing. Using a 'think aloud' approach, subjects were interviewed while completing DCE choice tasks. Constant comparison techniques were used to characterize the issues raised by subjects. Findings from the cognitive interviews were used to revise the initial DCE instruments.

RESULTS

Most subjects completed the DCE questionnaire without difficulty and considered two or more attributes in choosing between treatments. Two common patterns of decision-making emerged during the cognitive interviews: (1) counting 'pros' and 'cons' of each treatment alternative; and (2) using a 'rule-out' strategy to eliminate a given treatment choice if it included an undesirable attribute. Subjects routinely discounted the importance of certain attributes and, in a few cases, focused primarily on a single 'must-have' attribute.

CONCLUSION

Cognitive interviews provide valuable insights into the comprehension and interpretation of DCE attributes, the decision processes used by veterans during completion of choice tasks, and underlying reasons for non'-compensatory decision-making.

Family dynamics in a multi-ethnic Asian society: comparison of elderly CKD patients and their family caregivers experience with medical decision making for managing end stage kidney disease

Background

Elderly end stage kidney patients face a decision concerning whether or not to initiate dialysis. In Asia, this decision is highly influenced by family caregivers. The objective of this paper was to understand patients’ experience with and preferences for family involvement in treatment decisions, and via a series of hypothetical vignettes, to identify whether there was discordance in treatment preferences between patients and their caregivers, and how any potential conflicts were reconciled.

Methods

We conducted a survey with 151 elderly (aged ≥65) chronic kidney disease patients and their caregivers at outpatient renal clinics. The survey asked, when making treatment decisions, whom they wish makes the final decisions (i.e., preference) and who usually makes the final decisions (i.e., experience). The survey also presented a series of choice vignettes for managing patient’s condition and asked respondents to choose between two hypothetical treatment profiles in each vignette. Patients and caregivers were first interviewed separately in tandem, and then were brought together to choose a treatment jointly for vignettes where the initial treatment choice differed within the dyad. We used multivariate regressions to investigate the predictors of discordance and reconciliation.

Results

We found that most (51%) patients preferred and experienced (64%) significant involvement from caregivers. However, 38% of patients preferred to make final decisions alone but only 27% of patients did. In the hypothetical vignettes, caregivers chose the more intensive option (i.e., dialysis) more than patients did (26% vs 19%; p < 0.01). Overall, 44% of the dyads had discordance in at least 3 vignettes, and the odds of discordance within patient-caregiver dyads was higher when caregivers chose dialysis or treatment with the higher cost (p < 0.01). In half the cases, discordance resolved in the patients’ favor, and this was more likely to be the case if the patient was employed and wanted to be in charge of final decisions (p < 0.01).

Conclusions

Our results highlight the important role of caregivers in decision-making but also the potential for them to overstep. Clinicians should be aware of this challenge and identify strategies that minimize the chances that patients may receive treatments not consistent with their preferences.

Electronic supplementary material

The online version of this article (10.1186/s12882-019-1259-4) contains supplementary material, which is available to authorized users.

Parents' views on their children's use of eye drops and willingness to accept a new sustained-release subconjunctival injection

Aim

The objectives of this study were to explore parents’ views about their children’s use of regular eye drops and whether they would consider a sustained-release subconjunctival injection as a replacement for daily drops.

Methods

A survey was conducted with 134 parents of children with chronic eye diseases at the Singapore National Eye Centre. Parents were asked their views about their children’s use of eye drops and were then presented with a discrete choice experiment that, via a series of trade-off tasks, allowed for estimating demand for a series of hypothetical subconjunctival injections that varied along product features, including interval between administrations, risk of complications, out-of-pocket cost and whether it is recommended by the patient’s treating physician.

Results

Results showed that the vast majority of parents did not find administration of eye drops to be inconvenient (78%) nor did children complain about using daily eye drops (78%). Furthermore, only about half of parents whose child missed doses stated concerns about the consequences of non-compliance. The discrete choice experiment revealed that only one in five parents would consider a subconjunctival injection for their children. These parents tended to be more concerned about the consequences of non-compliance with eye drops, had children who administered the drops themselves or had other chronic disease requiring regular medication. Among these parents, risk of complications had the largest effect on injection uptake.

Conclusion

This study shows that parents do not find administration of daily eye drops to be a significant burden. As a result, most would not consider a subconjunctival injection unless risk of complications was extremely small.

Exploring how individuals complete the choice tasks in a discrete choice experiment: an interview study

Background

To be able to make valid inferences on stated preference data from a Discrete Choice Experiment (DCE) it is essential that researchers know if participants were actively involved, understood and interpreted the provided information correctly and whether they used complex decision strategies to make their choices and thereby acted in accordance with the continuity axiom.

Methods

During structured interviews, we explored how 70 participants evaluated and completed four discrete choice tasks aloud. Hereafter, additional questions were asked to further explore if participants understood the information that was provided to them and whether they used complex decision strategies (continuity axiom) when making their choices. Two existing DCE questionnaires on rotavirus vaccination and prostate cancer-screening served as case studies.

Results

A large proportion of the participants was not able to repeat the exact definition of the risk attributes as explained to them in the introduction of the questionnaire. The majority of the participants preferred more optimal over less optimal risk attribute levels. Most participants (66 %) mentioned three or more attributes when motivating their decisions, thereby acting in accordance with the continuity axiom. However, 16 out of 70 participants continuously mentioned less than three attributes when motivating their decision. Lower educated and less literate participants tended to mention less than three attributes when motivating their decision and used trading off between attributes less often as a decision-making strategy.

Conclusion

The majority of the participants seemed to have understood the provided information about the choice tasks, the attributes, and the levels. They used complex decision strategies (continuity axiom) and are therefore capable to adequately complete a DCE. However, based on the participants’ age, educational level and health literacy additional, actions should be undertaken to ensure that participants understand the choice tasks and complete the DCE as presumed.

Electronic supplementary material

The online version of this article (doi:10.1186/s12874-016-0140-4) contains supplementary material, which is available to authorized users.

Patients' & healthcare professionals' values regarding true- & false-positive diagnosis when colorectal cancer screening by CT colonography: discrete choice experiment

Purpose

To establish the relative weighting given by patients and healthcare professionals to gains in diagnostic sensitivity versus loss of specificity when using CT colonography (CTC) for colorectal cancer screening.

Materials and Methods

Following ethical approval and informed consent, 75 patients and 50 healthcare professionals undertook a discrete choice experiment in which they chose between “standard” CTC and “enhanced” CTC that raised diagnostic sensitivity 10% for either cancer or polyps in exchange for varying levels of specificity. We established the relative increase in false-positive diagnoses participants traded for an increase in true-positive diagnoses.

Results

Data from 122 participants were analysed. There were 30 (25%) non-traders for the cancer scenario and 20 (16%) for the polyp scenario. For cancer, the 10% gain in sensitivity was traded up to a median 45% (IQR 25 to >85) drop in specificity, equating to 2250 (IQR 1250 to >4250) additional false-positives per additional true-positive cancer, at 0.2% prevalence. For polyps, the figure was 15% (IQR 7.5 to 55), equating to 6 (IQR 3 to 22) additional false-positives per additional true-positive polyp, at 25% prevalence. Tipping points were significantly higher for patients than professionals for both cancer (85 vs 25, p<0.001) and polyps (55 vs 15, p<0.001). Patients were willing to pay significantly more for increased sensitivity for cancer (p = 0.021).

Conclusion

When screening for colorectal cancer, patients and professionals believe gains in true-positive diagnoses are worth much more than the negative consequences of a corresponding rise in false-positives. Evaluation of screening tests should account for this.