Mimicking Real-Life Decision Making in Health: Allowing Respondents Time to Think in a Discrete Choice Experiment

An Overview of Data Collection in Health Preference Research

This paper focuses on survey administration and data collection methods employed for stated-preference studies in health applications. First, it describes different types of survey administration methods, encompassing web-based surveys, face-to-face (in-person) surveys, and mail surveys. Second, the concept of sampling frames is introduced, clarifying distinctions between the target population and survey frame population. The discussion then extends to different types of sampling methods, such as probability and non-probability sampling, along with an evaluation of potential issues associated with different sampling methods within the context of health preference research. Third, the paper provides information about different recruitment methods, including web-surveys, leveraging patient groups, and in-clinic recruitment. Fourth, a crucial aspect addressed is the calculation of response rate, with insights into determining an adequate response rate and strategies to improve response rates in stated-preference surveys. Lastly, the paper concludes by discussing data management plans and suggesting insights for future research in this field. In summary, this paper examines the nuanced aspects of survey administration and data collection methods in stated-preference studies, offering valuable guidance for researchers and practitioners in the health domain.

Maximum Acceptable Risk Estimation Based on a Discrete Choice Experiment and a Probabilistic Threshold Technique

OBJECTIVE

We aimed to empirically compare maximum acceptable risk results estimated using both a discrete choice experiment (DCE) and a probabilistic threshold technique (PTT).

METHODS

Members of the UK general public (n = 982) completed an online survey including a DCE and a PTT (in random order) measuring their preferences for preventative treatment for rheumatoid arthritis. For the DCE, a Bayesian D-efficient design consisting of four blocks of 15 choice tasks was constructed including six attributes with varying levels. The PTT used identical risk and benefit attributes. For the DCE, a panel mixed-logit model was conducted, both mean and individual estimates were used to calculate maximum acceptable risk. For the PTT, interval regression was used to calculate maximum acceptable risk. Perceived complexity of the choice tasks and preference heterogeneity were investigated for both methods.

RESULTS

Maximum acceptable risk confidence intervals of both methods overlapped for serious infection and serious side effects but not for mild side effects (maximum acceptable risk was 32.7 percent-points lower in the PTT). Although, both DCE and PTT tasks overall were considered easy or very easy to understand and answer, significantly more respondents rated the DCE choice tasks as easier to understand compared with those who rated the PTT as easier (7-percentage point difference; p < 0.05).

CONCLUSIONS

Maximum acceptable risk estimate confidence intervals based on a DCE and a PTT overlapped for two out of the three included risk attributes. More respondents rated the DCE as easier to understand. This may suggest that the DCE is better suited in studies estimating maximum acceptable risk for multiple risk attributes of differing severity, while the PTT may be better suited when measuring heterogeneity in maximum acceptable risk estimates or when investigating one or more serious adverse events.

Suitability of Preference Methods Across the Medical Product Lifecycle: A Multicriteria Decision Analysis

OBJECTIVES

This study aimed to understand the importance of criteria describing methods (eg, duration, costs, validity, and outcomes) according to decision makers for each decision point in the medical product lifecycle (MPLC) and to determine the suitability of a discrete choice experiment, swing weighting, probabilistic threshold technique, and best-worst scale cases 1 and 2 at each decision point in the MPLC.

METHODS

Applying multicriteria decision analysis, an online survey was sent to MPLC decision makers (ie, industry, regulatory, and health technology assessment representatives). They ranked and weighted 19 methods criteria from an existing performance matrix about their respective decisions across the MPLC. All criteria were given a relative weight based on the ranking and rating in the survey after which an overall suitability score was calculated for each preference elicitation method per decision point. Sensitivity analyses were conducted to reflect uncertainty in the performance matrix.

RESULTS

Fifty-nine industry, 29 regulatory, and 5 health technology assessment representatives completed the surveys. Overall, "estimating trade-offs between treatment characteristics" and "estimating weights for treatment characteristics" were highly important criteria throughout all MPLC decision points, whereas other criteria were most important only for specific MPLC stages. Swing weighting and probabilistic threshold technique received significantly higher suitability scores across decision points than other methods. Sensitivity analyses showed substantial impact of uncertainty in the performance matrix.

CONCLUSION

Although discrete choice experiment is the most applied preference elicitation method, other methods should also be considered to address the needs of decision makers. Development of evidence-based guidance documents for designing, conducting, and analyzing such methods could enhance their use.

Impact of information and deliberation on the consistency of preferences for prioritization in health care - evidence from discrete choice experiments undertaken alongside citizens' juries

BACKGROUND

Public preferences are an important consideration for priority-setting. Critics suggest preferences of the public who are potentially naïve to the issue under consideration may lead to sub-optimal decisions. We assessed the impact of information and deliberation via a Citizens' Jury (CJ) or preference elicitation methods (Discrete Choice Experiment, DCE) on preferences for prioritizing access to bariatric surgery.

METHODS

Preferences for seven prioritization criteria (e.g. obesity level, obesity-related comorbidities) were elicited from three groups who completed a DCE: (i) participants from two CJs (n = 28); (ii) controls who did not participate in the jury (n = 21); (iii) population sample (n = 1,994). Participants in the jury and control groups completed the DCE pre- and post-jury. DCE data were analyzed using multinomial logit models to derive "priority weights" for criteria for access to surgery. The rank order of criteria was compared across groups, time points and CJ recommendations.

RESULTS

The extent to which the criteria were considered important were broadly consistent across groups and were similar to jury recommendations but with variation in the rank order. Preferences of jurors but not controls were more differentiated (that is, criteria were assigned a greater range of priority weights) after than before the jury. Juror preferences pre-jury were similar to that of the public but appeared to change during the course of the jury with greater priority given to a person with comorbidity. Conversely, controls appeared to give a lower priority to those with comorbidity and higher priority to treating very severe obesity after than before the jury.

CONCLUSION

Being informed and undertaking deliberation had little impact on the criteria that were considered to be relevant for prioritizing access to bariatric surgery but may have a small impact on the relative importance of criteria. CJs may clarify underlying rationale but may not provide substantially different prioritization recommendations compared to a DCE.

Contingent Valuation: A Pilot Study for Eliciting Willingness to Pay for a Reduction in Mortality From Vaccine-Preventable Illnesses for Children and Adults in Bangladesh

OBJECTIVES

The contingent valuation (CV) method elicits willingness to pay (WTP) for calculating the value of statistical life (VSL). CV approaches for assessing VSL are uncommon in many low and middle-income countries (LMICs). Between 2008 and 2018 only 44 articles utilized WTP in a health-related field and of these only 5 (11%) utilized CV to assess the WTP for a mortality risk reduction. We elicit WTP estimates and compute VSL using the CV method in Bangladesh.

METHODS

The pilot study was primarily aimed at developing best practice guidelines for CV studies in LMICs to get more robust WTP estimates. To this end, we explored three methodological a) Varying the name of the intervention, keeping all other characteristics constant; b) varying the effectiveness of the health intervention and c) offering an overnight period to think about the WTP scenario. The survey was administered 413 randomly selected participants. VSL was for a 1/3000 mortality risk reduction.

RESULTS

We had more males (54%) than females (46%) and the mean annual self-reported income was $5,683.36. Mean VSL is $11,339.70 with a median of $10,413. The ratio of child: adult WTP is approximately 1 by both gender and age category. The vaccine intervention had the largest amount of $0 WTP and protest responses (52% and 58% respectively). 93% of the participants were able to describe (teach-back) the vaccine effectiveness using their own family as an example.

CONCLUSION

Our study provides empirical evidence on how to better generate CV surveys to produce more robust WTP estimates.

Opportunities and Challenges of Web-Based and Remotely Administered Surveys for Patient Preference Studies in a Vulnerable Population

The application of web-based and remotely administered surveys is becoming increasingly popular due to the fact that it offers numerous advantages over traditional paper-based or computer-based surveys completed in the presence of the researcher. However, it is unclear whether complex preference elicitation tasks administered online in highly vulnerable patient populations are also feasible. This commentary discusses opportunities and challenges of conducting quantitative patient preference studies in lung cancer patients using web-based modes of data collection. We refer to our recent experience in the context of the Patient Preference in Benefit-Risk Assessments during the Drug Life Cycle (PREFER) project. Among the main advantages were the possibility of reaching a wider and geographically distant population in a shorter timeframe while reducing the financial costs of testing, the greater flexibility offered and the reduced burden on the patients. Some limitations were also identified and should be the object of further research, including the potential lack of inclusiveness of the research, the lack of control over who is completing the survey, a poor comprehension of the study material, and ultimately a lower level of engagement with the study. Despite these limitations, experience from the PREFER project suggests that online quantitative methods for data collection may provide a valuable method to explore preferences in vulnerable patient populations beyond the COVID-19 pandemic.