A think aloud study comparing the validity and acceptability of discrete choice and best worst scaling methods

Attribute Development in Health-Related Discrete Choice Experiments: A Systematic Review of Qualitative Methods and Techniques to Inform Quantitative Instruments

OBJECTIVES

This review sought to identify the qualitative methods and techniques that researchers have used in the past decade to develop attributes and inform health-related discrete choice experiments (DCEs) surveys from a patient perspective.

METHODS

The review followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines for reporting systematic reviews. An adapted appraisal tool following guidelines for reporting qualitative research for quantitative instruments and criteria for attribute development in DCEs was applied for quality assessment and data extraction. A narrative approach was used to synthesize data. This examination included consideration of issues pertaining to sampling, data collection, data analysis, attribute list reduction, wording, methodological adaptations to capture patient preferences, and testing the pre-experimental design decisions of the DCE survey.

RESULTS

Of 8505 articles identified for abstract screening, 680 were included for full-text screening, 36 of which met the inclusion criteria. Practices to improve methodological robustness included pre-data collection materials to inform instruments, data collection methods specific for decision-making scenarios, purposeful selection of data analysis methods to address the research question, and participants' involvement in reducing the list of attributes. Examples of methodological adaptations for patients were noted.

CONCLUSIONS

DCEs have the potential to become a mixed-method approach in which the qualitative phase informs a reduced list of attributes for a survey, serves the predesign decisions of the experiment by testing trade-offs, overlapping, understandability, face, and content validity and provides explanations of the quantitative results. Establishing guidelines for using qualitative methods for DCE attribute development may help to broadly enhance the methodological robustness of DCEs.

Putting the Choice in Choice Tasks: Incorporating Preference Elicitation Tasks in Health Preference Research

Choice-based preference elicitation methods such as the discrete choice experiment (DCE) present hypothetical choices to respondents, with an expectation that these hypothetical choices accurately reflect a 'real world' health-related decision context and that consequently the choice data can be held to be a true representation of the respondent's health or treatment preferences. For this to be the case, careful consideration needs to be given to the format of the choice task in a choice experiment. The overarching aim of this paper is to highlight important aspects to consider when designing and 'setting up' the choice tasks to be presented to respondents in a DCE. This includes the importance of considering the potential impact of format (e.g. choice context, choice set presentation and size) as well as choice set content (e.g. labelled and unlabelled choice sets and inclusion of reference alternatives) and choice questions (stated choice versus additional questions designed to explore complete preference orders) on the preference estimates that are elicited from studies. We endeavoure to instil a holistic approach to choice task design that considers format alongside content, experimental design and analysis.

Navigating Public Policy Responses to a Pandemic: The Balancing Act Between Physical Health, Mental Health, and Household Income

OBJECTIVE

During COVID-19, governments imposed restrictions that reduced pandemic-related health risks but likely increased personal and societal mental health risk, partly through reductions in household income. This study aimed to quantify the public's willingness to accept trade-offs between pandemic health risks, household income reduction, and increased risk of mental illness that may result from future pandemic-related policies.

METHODS

A total of 547 adults from an online panel participated in a discrete choice experiment where they were asked to choose between hypothetical future pandemic scenarios. Each scenario was characterized by personal and societal risks of dying from the pandemic, experiencing long-term complications, developing anxiety/depression, and reductions in household income. A latent class regression was used to estimate trade-offs.

RESULTS

Respondents state a willingness to make trade-offs across these attributes if the benefits are large enough. They are willing to accept 0.8% lower household income (0.7-1.0), 2.7% higher personal risk of anxiety/depression (1.8-3.6), or 3.2% higher societal rate of anxiety/depression (1.7-4.7) in exchange for 300 fewer deaths from the pandemic.

CONCLUSION

Results reveal that individuals are willing to accept lower household income and higher rates of mental illness, both personal and societal, if the physical health benefits are large enough. Respondents placed greater emphasis on maintaining personal, as opposed to societal, mental health risk and were most interested in preventing pandemic-related deaths. Governments should consider less restrictive policies when pandemics have high morbidity but low mortality to avoid the prospect of improving physical health while simultaneously reducing net social welfare.

An Overview of Data Collection in Health Preference Research

This paper focuses on survey administration and data collection methods employed for stated-preference studies in health applications. First, it describes different types of survey administration methods, encompassing web-based surveys, face-to-face (in-person) surveys, and mail surveys. Second, the concept of sampling frames is introduced, clarifying distinctions between the target population and survey frame population. The discussion then extends to different types of sampling methods, such as probability and non-probability sampling, along with an evaluation of potential issues associated with different sampling methods within the context of health preference research. Third, the paper provides information about different recruitment methods, including web-surveys, leveraging patient groups, and in-clinic recruitment. Fourth, a crucial aspect addressed is the calculation of response rate, with insights into determining an adequate response rate and strategies to improve response rates in stated-preference surveys. Lastly, the paper concludes by discussing data management plans and suggesting insights for future research in this field. In summary, this paper examines the nuanced aspects of survey administration and data collection methods in stated-preference studies, offering valuable guidance for researchers and practitioners in the health domain.

Eliciting Depression Patients' Preferences for Medication Management: A Protocol for Discrete Choice Experiment

Introduction

Depression threatens people's lives and imposes huge economic burden. Antidepressant therapy is the first-line treatment for depression, and patient adherence to medication is the key to successful treatment. Depression patients have poor medication adherence, which leads to failure of depression management and significantly poorer clinical outcomes. Incorporating patient preferences into clinical decisions can improve uptake rates, optimize treatment adherence. A discrete choice experiment (DCE) can elicit and quantify individual preferences. Previous DCE studies were conducted in developed countries and ignored the influences of factors other than the medication. This paper outlines an ongoing DCE that aims to (1) explore medication-management-related characteristics that may affect depression patients' adherence to antidepressant, (2) elicit how depression patients consider the trade-offs among different medication managements.

Methods

The six attributes and their levels were developed through a literature review, semi-structured interviews and experts and focus group discussions. A fractional factorial design in the software Ngene 1.2 version was used to generate 36 choice sets, and they were divided into 3 blocks. A mixed logit model will be used to explore the patients' preferences, willingness to pay and uptake rate of depression patients for medication management attributes.

Results

The final questionnaire consists of three parts. The first is the introduction, which introduces the purpose of the study and the requirements of completing the questionnaire. This was followed by a general information questionnaire, which included sociodemographic characteristics. The last part is DCE tasks, which include 13 DCE choice sets, and each choice set include two alternative and one "opt-out" option. The pilot-test results showed the questionnaire was easy to understand and could be used in formal surveys.

Conclusion

Our study shows how the development process of the study can be conducted and reported systematically and rigorously according to the theoretical foundation and design principles in DCE.

Stated-Preference Survey Design and Testing in Health Applications

Following the conceptualization of a well-formulated and relevant research question, selection of an appropriate stated-preference method, and related methodological issues, researchers are tasked with developing a survey instrument. A major goal of designing a stated-preference survey for health applications is to elicit high-quality data that reflect thoughtful responses from well-informed respondents. Achieving this goal requires researchers to design engaging surveys that maximize response rates, minimize hypothetical bias, and collect all the necessary information needed to answer the research question. Designing such a survey requires researchers to make numerous interrelated decisions that build upon the decision context, selection of attributes, and experimental design. Such decisions include considering the setting(s) and study population in which the survey will be administered, the format and mode of administration, and types of contextual information to collect. Development of a survey is an interactive process in which feedback from respondents should be collected and documented through qualitative pre-test interviews and pilot testing. This paper describes important issues to consider across all major steps required to design and test a stated-choice survey to elicit patient preferences for health preference research.

Unravelling the Self-Report Versus Proxy-Report Conundrum for Older Aged Care Residents: Findings from a Mixed-Methods Study

OBJECTIVES

No guidance currently exists as to the cognition threshold beyond which self-reported quality of life for older people with cognitive impairment and dementia is unreliable.

METHODS

Older aged care residents (≥ 65 years) were randomly assigned to complete the EQ-5D-5L in computer-based (eye movements were tracked) or hard copy (participants were encouraged to 'think aloud') format. Cognition was assessed using the Mini-Mental State Examination (MMSE). Think aloud and eye tracking data were analysed by two raters, blinded to MMSE scores. At the participant level, predefined criteria were used to assign traffic light grades (green, amber, red). These grades indicate the extent to which extracted data elements provided evidence of self-report reliability. The MMSE-defined cognition threshold was determined following review of the distributions of assigned traffic light grades.

RESULTS

Eighty-one residents participated and provided complete data (38 eye tracking, 43 think aloud). In the think aloud cohort, all participants with an MMSE score ≤ 23 (n = 10) received an amber or red grade, while 64% of participants with an MMSE score ≥ 24 (21 of 33) received green grades. In the eye tracking cohort, 68% of participants with an MMSE score ≥ 24 (15 of 22) received green grades. Of the 16 eye tracking participants with an MMSE score ≤ 23, 14 (88%) received an amber or red grade.

CONCLUSIONS

Most older residents with an MMSE score ≥ 24 have sufficient cognitive capacity to self-complete the EQ-5D-5L. More research is needed to better understand self-completion reliability for other quality-of-life instruments in cognitively impaired populations.

Trade-Offs between Vaccine Effectiveness and Vaccine Safety: Personal versus Policy Decisions

OBJECTIVE

We aimed to investigate whether individuals' trade-offs between vaccine effectiveness and vaccine safety vary if they are asked to consider the perspective of a policymaker making decisions for others compared with the decisions they would make for themselves.

METHOD

A web-enabled discrete choice experiment survey was administered between 1 April and 1 May 2022 to participants recruited from the general population of two Southeast Asian countries (Indonesia and Vietnam). In each country, 500 participants were randomly assigned to make decisions regarding coronavirus disease 2019 (COVID-19) vaccines for others as a policymaker or in a personal capacity for their own use. Vaccines were characterized by three attributes: (1) effectiveness of the vaccine in reducing infection rate; (2) effectiveness of the vaccine in reducing hospitalization among those infected; and (3) risk of death from vaccine-related serious adverse events. A mixed logit model was utilized for analyses.

RESULTS

Based on the attributes and levels used in this study, the most important vaccine attribute was the risk of death from vaccine-related adverse events, followed by effectiveness in reducing infection rate and hospitalizations. Compared with personal decisions, the mean probability of choosing a vaccine was (1) lower, and (2) more sensitive to the changes in risk of death from adverse events in policy decisions (p ≤ 0.01).

CONCLUSIONS AND RELEVANCE

Our results suggest that, in the face of an infectious disease pandemic, individuals are likely to be more risk-averse to vaccine-related deaths when making decisions for others as a policymaker than they would for themselves.

Preferences for Enhanced Primary Care Services Among Older Individuals and Primary Care Physicians

OBJECTIVE

We aimed to identify the factors that are most important for community-dwelling older individuals (i.e., users) and primary care (PC) providers to enhance PC services.

METHODS

Discrete choice experiment surveys were administered to 747 individuals aged ≥ 60 years and 242 PC physicians in Singapore between December 2020 and August 2021. Participants were asked to choose between two hypothetical PC clinics and their current clinic. Latent class models were used to estimate the relative attribute importance (RAI) and to calculate the predicted uptake for enhanced PC services.

RESULTS

Based on the attributes and levels used in this study, the out-of-pocket cost (RAI: 47%) and types of services offered (RAI: 25%) were the most important attributes for users while working hours (RAI: 28%) and patient load (RAI: 25%) were the most important for providers. For out-of-pocket visit costs ranging from Singapore dollars (S)$100 to S$5, users' predicted uptake for enhanced PC services ranged from 46 to 84%. For daily patient loads ranging from 60 to 20 patients, providers' predicted uptake ranged from 64 to 91%, assuming their income remains unchanged.

CONCLUSIONS

Our study provides timely insights for the development of strategies to support the government's new health care initiative (HealthierSG), which places PC at the center of Singapore's healthcare system. The ability to choose their preferred clinic, low out-of-pocket costs and types of services offered (for users), and reasonable working conditions (for providers) were the key factors for users and providers to participate in enhanced PC services.

Understanding the valuation of paediatric health-related quality of life: a qualitative study protocol

INTRODUCTION

There is evidence from previous studies that adults value paediatric health-related quality of life (HRQoL) and adult HRQoL differently. Less is known about how adolescents value paediatric HRQoL and whether their valuation and decision-making processes differ from those of adults. Discrete choice experiments (DCEs) are widely used to develop value sets for measures of HRQoL, but there is still much to understand about whether and how the methods choices in the implementation of DCE valuation tasks, such as format, presentation and perspective, affect the decision-making process of participants. This paper describes the protocol for a qualitative study that aims to explore the decision-making process of adults and adolescents when completing DCE valuation tasks. The study will also explore the impact of methodological choices in the design of DCE studies (including decisions about format and presentation) on participants' thinking process.

METHODS AND ANALYSIS

An interview protocol has been developed using DCE valuation tasks. Interviews will be conducted online via Zoom with both an adolescent and adult sample. In the interview, the participant will be asked to go through some DCE valuation tasks while 'thinking aloud'. After completion of the survey, participants will then be asked some predetermined questions in relation to various aspects of the DCE tasks. Interviews will be recorded and transcribed and analysed using a thematic analysis approach.

ETHICS AND DISSEMINATION

Ethics approval for this study has been received for the adult sample (UTS ETH20-9632) as well as the youth sample (UTS ETH22-6970) from the University of Technology Sydney Human Research Ethics Committee. Results from this study will inform the methods to be used in development of value sets for use in the health technology assessment of paediatric interventions and treatments. Findings from this study will also be disseminated through national/international conferences and peer-reviewed journals.

Towards machine learning for moral choice analysis in health economics: A literature review and research agenda

BACKGROUND

Discrete choice models (DCMs) for moral choice analysis will likely lead to erroneous model outcomes and misguided policy recommendations, as only some characteristics of moral decision-making are considered. Machine learning (ML) is recently gaining interest in the field of discrete choice modelling. This paper explores the potential of combining DCMs and ML to study moral decision-making more accurately and better inform policy decisions in healthcare.

METHODS

An interdisciplinary literature search across four databases - PubMed, Scopus, Web of Science, and Arxiv - was conducted to gather papers. Based on the Preferred Reporting Items for Systematic and Meta-analyses (PRISMA) guideline, studies were screened for eligibility on inclusion criteria and extracted attributes from eligible papers. Of the 6285 articles, we included 277 studies.

RESULTS

DCMs have shortcomings in studying moral decision-making. Whilst the DCMs' mathematical elegance and behavioural appeal hold clear interpretations, the models do not account for the 'moral' cost and benefit in an individual's utility calculation. The literature showed that ML obtains higher predictive power, model flexibility, and ability to handle large and unstructured datasets. Combining the strengths of ML methods with DCMs has the potential for studying moral decision-making.

CONCLUSIONS

By providing a research agenda, this paper highlights that ML has clear potential to i) find and deepen the utility specification of DCMs, and ii) enrich the insights extracted from DCMs by considering the intrapersonal determinants of moral decision-making.

Preferences of older adults for healthcare models designed to improve care coordination: evidence from Western Switzerland

Implementing innovations in care delivery in Switzerland is challenging due to the fragmented nature of the system and the specificities of the political process (i.e., direct democracy, decentralized decision-making). In this context, it is particularly important to account for population preferences when designing policies. We designed a discrete choice experiment to study population preferences for coordination-improving care models. Specifically, we assessed the relative importance of model characteristics (i.e., insurance premium, presence of care coordinator, access to specialists, use of EMR, cost-sharing for chronic patients, incentives for informal care), and predicted uptake under different policy scenarios. We accounted for heterogeneity in preferences for the status quo option using an error component logit model. Respondents attached the highest importance to the price attribute (i.e. insurance premium) (0.31, CI: 0.27- 0.36) and to the presence of a care coordinator (0.27, CI: 0.23 - 0.31). Policy scenarios showed for instance that gatekeeping would be preferred to free access to specialists if the model includes a GP or an interprofessional team as a care coordinator. Although attachment to the status quo is high in the studied population, there are potential ways to improve acceptance of alternative care models by implementation of positively valued innovations.

Stroke survivors' preferences for post-stroke self-management programs: A discrete choice experiment

BACKGROUND AND PURPOSE

Self-management programs enhance survival in stroke patients. However, they require patient-centered designs to be effective. The aim of this study was therefore to investigate the type of post-stroke self-management programs that appeal to stroke survivors, and to estimate their willingness to participate in such programs.

METHODS

A Discrete Choice Experiment was administered to patients who had either a transient ischemic attack (TIA) or stroke within the past 3 years and were cognitively intact (i.e., stroke survivors). Stroke survivors were presented with eight choice tasks and asked to choose between 'No Program' and two hypothetical post-stroke management programs that varied by six attributes: Topics covered by the program; schedule of the program; frequency and duration of the sessions; number of participants; out-of-pocket registration fee for the whole program; and rewards for completing the program.

RESULTS

The analysis involved 146 stroke survivors. Based on the mixed logit model, the predicted willingness to participate ranged from 53% to 76%. The most popular characteristics in a program were topics on health education and risk management, being scheduled during weekends as four sessions that are each 2 hours long and involve four participants, a registration fee of SGD50 (∼USD36), and SGD500 (∼USD359) reward for program completion.

CONCLUSIONS

Interest in post-stroke self-management programs was high, with at least half of the sample showing interest in participating in these programs. Program features such as focusing on health education and risk management, charging a low registration fee, and offering incentives helped to increase the demand.

Comparing Outcomes of a Discrete Choice Experiment and Case 2 Best-Worst Scaling: An Application to Neuromuscular Disease Treatment

BACKGROUND AND OBJECTIVES

Case 2 best-worst scaling (BWS-2) is an increasingly popular method to elicit patient preferences. Because BWS-2 potentially has a lower cognitive burden compared with discrete choice experiments, the aim of this study was to compare treatment preference weights and relative importance scores.

METHODS

Patients with neuromuscular diseases completed an online survey at two different moments in time, completing one method per occasion. Patients were randomly assigned to either first a discrete choice experiment or BWS-2. Attributes included: muscle strength, energy endurance, balance, cognition, chance of blurry vision, and chance of liver damage. Multinomial logit was used to calculate overall relative importance scores and latent class logit was used to estimate heterogeneous preference weights and to calculate the relative importance scores of the attributes for each latent class.

RESULTS

A total of 140 patients were included for analyses. Overall relative importance scores showed differences in attribute importance rankings between a discrete choice experiment and BWS-2. Latent class analyses indicated three latent classes for both methods, with a specific class in both the discrete choice experiment and BWS-2 in which (avoiding) liver damage was the most important attribute. Ex-post analyses showed that classes differed in sex, age, level of education, and disease status. The discrete choice experiment was easier to understand compared with BWS-2.

CONCLUSIONS

This study showed that using a discrete choice experiment and BWS-2 leads to different outcomes, both in preference weights as well as in relative importance scores, which might have been caused by the different framing of risks in BWS-2. However, a latent class analysis revealed similar latent classes between methods. Careful consideration about method selection is required, while keeping the specific decision context in mind and pilot testing the methods.

Understanding patient cost-sharing thresholds for diabetes treatment attributes via a discrete choice experiment

BACKGROUND: The process used to prefer certain products across drug classes for diabetes is generally focused on comparative effectiveness and cost. However, payers rarely tie patient preference for treatment attributes to formulary management resulting in a misalignment of value defined by providers, payers, and patients. OBJECTIVES: To explore patients' willingness to pay (WTP) for the predetermined high-value and low-value type 2 diabetes mellitus (T2DM) treatments within a health plan. METHODS: A cross-sectional discrete choice experiment (DCE) survey was used to determine patient preference for the benefit, risk, and cost attributes of T2DM treatments. A comprehensive literature review of patient preference studies in diabetes and a review of guidelines and medical literature identified study attributes. Patients and diabetes experts were interviewed and instructed to identify, prioritize, and comment on which attributes of diabetes treatments were most important to T2DM patients. The patients enrolled in a health plan were asked to respond to the survey. A multinomial logit model was developed to determine the relative importance and the patient's WTP of each attribute. The patients' relative values based on WTPs for T2DM treatments were calculated and compared with the treatments by a health plan. RESULTS: A total of 7 attributes were selected to develop a web-based DCE questionnaire survey. The responses from a total of 58 patients were analyzed. Almost half (48.3%) of the respondents took oral medications and injections for T2DM. The most prevalent side effects due to diabetes medications were gastrointestinal (43.1%), followed by weight gain (39.7%) and nausea (32.8%). Patients were willing to pay more for treatments with proven cardiovascular benefit and for the risk reduction of hospitalization from heart failure. On the other hand, they would pay less for treatments with higher gastrointestinal side effects. Patients were willing to pay the most for sodium-glucose cotransporter 2 inhibitor and glucagon-like peptide 1 receptor agonist agents and the least for dipeptidyl peptidase-4 inhibitors and thiazolidinediones. CONCLUSIONS: This study provides information to better align patient, provider, and payer preferences in both benefit design and value-based formulary strategy for diabetes treatments. A preferred placement of treatments with cardiovascular benefits and lower adverse gastrointestinal side effects may lead to increased adherence to medications and improved clinical outcomes at a lower overall cost to both patients and their health plan. DISCLOSURES: This study was supported by a grant from the PhRMA Foundation.

Deciding Between SF-6Dv2 Health States: A Think-Aloud Study of Decision-Making Strategies Used in Discrete Choice Experiments

OBJECTIVE

This study aimed to gain insight into decision-making strategies individuals used when evaluating pairs of SF-6Dv2 health states in discrete choice experiments (DCEs).

METHODS

This qualitative, cross-sectional, noninterventional study asked participants to use a think-aloud approach to compare SF-6Dv2 health states in DCEs. Thematic analysis focused on comprehension and cognitive strategies used to compare health states and make decisions.

RESULTS

Participants (N = 40) used 3 main strategies when completing DCEs: (1) trading, (2) reinterpretation, and (3) relying on previous experience. Trading was the most common strategy, used by everyone at least once, and involved prioritizing key attributes, such as preferring a health state with significant depression but no bodily pain. Reinterpretation was used by 17 participants and involved reconstructing health states by changing underlying assumptions (eg, rationalizing selecting a health state with significant pain because they could take pain medications). Finally, some (n = 13) relied on previous experience when making decisions on some choice tasks. Participants with experience dealing with pain, for instance, prioritized health states with the least impact in this dimension.

CONCLUSIONS

Qualitatively evaluating the decision-making strategies used in DCEs allows researchers to evaluate whether the tasks and attributes are interpreted accurately. The findings from this study add to the understanding of the generation of SF-6Dv2 health utility weights and the validity of these weights (e.g., reinterpreting health states could undermine the validity of DCEs and utility weights), and the overall usefulness of the SF-6Dv2. The methodology described in this study can and should be carried forth in valuing other health utility measures, not just the SF-6Dv2.

Using discrete choice experiments to elicit the service preferences of people with mild intellectual disability: An exploratory study

The National Disability Insurance Scheme (NDIS) in Australia and other similar international movements towards consumer direction have highlighted the importance of including consumers to ensure their service preferences are operationalised. Discrete Choice Experiments (DCEs) are an established method to quantify consumer preferences. The feasibility of using DCEs with people with intellectual disability is largely untested. Consenting participants eligible for disability support services (n = 18) participated in the mixed methods exploratory study. The DCE comprised a series of choices between two hypothetical service providers offering a combination of services relating to social and economic participation (e.g. support with finding and keeping a job), with four levels of service (no service, online support, group support, one to one support). Pictographs and simplified English were used to represent the hypothetical services and levels and a 'think aloud' protocol adopted. Most participants (N = 16, 89%) completed the DCE task. The findings from the think aloud task indicated that some participants were weighing up the options and making decisions based on their goals and personal preferences. However, other participants did not focus on all presented attributes and levels when making a decision; a common 'short-cut' heuristic also observed in DCE tasks with general population participants. Further research including investigating other DCE techniques, such as best-worst scaling, would be beneficial to identify how preference-elicitation tasks can be developed and applied with people with intellectual disabilities to ensure that future service innovations are designed and administered in ways which best meet their needs and preferences.

Do we agree or disagree? A systematic review of the application of preference-based instruments in self and proxy reporting of quality of life in older people

PURPOSE

Quality of life is an important person-centred outcome in health and aged care settings. Due to an increasing prevalence of cognitive decline and dementia in ageing populations, a proportion of older people receiving health and aged care services may not be able to reliably assess their own quality of life, highlighting the need for proxy assessment. This systematic review sought to investigate the level of agreement between self and proxy-report of older people's quality of life using established preference-based instruments of quality of life suitable for economic evaluation.

METHODS

A systematic review was conducted following PRISMA guidelines. Eight databases were searched: Web of Science, Scopus, Medline, Econlit, PsychINFO, CINAHL, Ageline and Cochrane Library. Information was extracted on the instruments, population samples (including any cognitive thresholds applied), mean scores, type of proxy, and measures of inter-rater agreement.

RESULTS

A total of 50 studies using eight different preference-based quality of life instruments were identified. Most studies were cross-sectional (72%) with a wide variety of cognitive assessments and thresholds applied to define older participants with cognitive impairment. The most common proxies were family members, mostly spouses. The level of agreement between self and proxy-report was generally poor - irrespective of the instrument applied or type of proxy - with proxy-report generally indicating lower levels of quality of life than self-report. There was some evidence of stronger agreement on more observable quality of life domains e.g., physical health and mobility, relative to less observable domains e.g. emotional well-being. Few studies tracked self and/or proxy-report of quality of life longitudinally.

CONCLUSIONS

More research is needed to develop evidence to inform guidance on self-report versus proxy-report of quality of life for older people receiving health and aged care services. Until then, the collection of both self and proxy reports as complementary measures is indicated.

The Preferences for Attributes of Initial COVID-19 Diagnosis in the US and China during the Pandemic: A Discrete Choice Experiment with Propensity Score Matching

Background

China and the United States play critical leading roles in the global effort to contain the COVID-19 virus. Therefore, their population’s preferences for initial diagnosis were compared to provide policy and clinical insights.

Objective

We aim to quantify and compare the public’s preferences for medical management of fever and the attributes of initial diagnosis in the case of presenting symptoms during the COVID-19 pandemic in China and the United States.

Methods

We conducted a cross-sectional study from January to March 2021 in China and the United States using an online discrete choice experiment (DCE) questionnaire distributed through Amazon Mechanical Turk (MTurk; in the United States) and recruited volunteers (in China). Propensity score matching (PSM) was used to match the 2 groups of respondents from China and the United States to minimize confounding effects. In addition, the respondents’ preferences for different diagnosis options were evaluated using a mixed logit model (MXL) and latent class models (LCMs). Moreover, demographic data were collected and compared using the chi-square test, Fisher test, and Mann-Whitney U test.

Results

A total of 9112 respondents (5411, 59.4%, from China and 3701, 40.6%, from the United States) who completed our survey were included in our analysis. After PSM, 1240 (22.9%) respondents from China and 1240 (33.5%) from the United States were matched for sex, age, educational level, occupation, and annual salary levels. The segmented sizes of 3 classes of respondents from China were 870 (70.2%), 270 (21.8%), and 100 (8.0%), respectively. Meanwhile, the US respondents’ segmented sizes were 269 (21.7%), 139 (11.2%), and 832 (67.1%), respectively. Respondents from China attached the greatest importance to the type of medical institution (weighted importance=40.0%), while those from the United States valued the waiting time (weighted importance=31.5%) the most. Respondents from China preferred the emergency department (coefficient=0.973, reference level: online consultation) and fever clinic (a special clinic for the treatment of fever patients for the prevention and control of acute infectious diseases in China; coefficient=0.974, reference level: online consultation), while those from the United States preferred private clinics (general practices; coefficient=0.543, reference level: online consultation). Additionally, shorter waiting times, COVID-19 nucleic acid testing arrangements, higher reimbursement rates, and lower costs were always preferred.

Conclusions

Improvements in the availability of COVID-19 testing and medical professional skills and increased designated health care facilities may help boost potential health care seeking during COVID-19 and prevent unrecognized community spreading of SARS-CoV-2 in China and the United States. Moreover, to better prevent future waves of pandemics, identify undiagnosed patients, and encourage those undiagnosed to seek health care services to curb the pandemic, the hierarchical diagnosis and treatment system needs improvement in China, and the United States should focus on reducing diagnosis costs and raising the reimbursement rate of medical insurance.

Patient Medication Preferences for Managing Dry Eye Disease: The Importance of Medication Side Effects

Objectives

The side effects of dry eye medications can lead to medication non-adherence and, eventually, to poor outcomes. This study aimed to quantify to what extent the side effects of dry eye disease (DED) medications (burning/stinging sensation and blurring) are important to patients compared to medication benefits or costs.

Methods

Patients diagnosed with DED were recruited at a referral eye center in Singapore (n = 139). This study utilized a Discrete Choice Experiment where patients were presented with 10 choice tasks where they were asked to choose between their current medication (or no medication), and two hypothetical medications that varied based on five attributes: duration of burning/stinging, duration of blurring, time to medication effectiveness, medication frequency, and out-of-pocket cost. The main outcomes were relative attribute importance and predicted uptake.

Results

Latent class logistic regressions found two groups with distinct preferences. For both classes, duration of burning/stinging (Class 1 = 23%, Class 2 = 29%) and cost (Class 1 = 24%, Class 2 = 27%) were the most important attributes while duration of blurring (Class 1 = 15%, Class 2 = 9%) was the least important. The predicted uptake of a medication increased 18 percentage-points when burning/stinging duration decreased from 2 h to a few minutes. The predicted uptake for new medications was lowest for those on medication with well-controlled symptoms and highest for those who were not on medication and could not control their symptoms effectively.

Conclusion

This study showed that duration of burning/stinging was an important factor when choosing medications. Incorporating patient preferences in medication decisions can potentially improve patient acceptance of a treatment regimen.

Supplementary Information

The online version contains supplementary material available at 10.1007/s40271-022-00586-8.

Parturients' Stated Preferences for Labor Analgesia: A Discrete Choice Experiment

OBJECTIVE

The objective was to investigate the extent to which treatment benefits, risks and costs affected parturients' preferences for labor analgesia.

METHODS

We recruited 248 healthy parturients prior to labor at an antenatal ward and administered a discrete choice experiment survey. Parturients were asked to choose among four hypothetical forms of labor analgesia: epidural analgesia, pethidine, Entonox and no analgesia, which were defined by: pain score, duration of second stage of labor, risks of instrumental delivery, back pain and permanent nerve injury, and out-of-pocket cost. We used mixed logit model to calculate the relative importance of each attribute (out of 100).

RESULTS

Parturients preferred receiving labor analgesia over not receiving analgesia and those who had positive past experience with epidural preferred epidural over other modalities. Out-of-pocket cost (28%), duration of second stage of labor (26%) and pain score following treatment (18%) were the most important attributes.

CONCLUSION

Out-of-pocket cost was a major concern. Parturients prioritized having lower pain and shorter labor experience over risks associated with epidural analgesia. Parturients should be presented with realistic range of risks of side-effects so that they can decide how to balance risks against benefits and costs associated with child labor.

Developing a preference-based measure for weight-specific health-related quality of life in adolescence: the WAItE UK valuation study protocol

INTRODUCTION

Adolescent obesity is a public health problem in the UK. The Weight-Specific Adolescent Instrument for Economic Evaluation (WAItE) has been developed as the first weight-specific health-related quality of life measure appropriate for economic evaluation, but currently cannot be used to generate quality-adjusted life years (QALYs), which are the basis of cost-utility analysis. Generic measures (such as the EQ-5D-Y or CHU-9D) may be insensitive to small but important health changes in overweight or obese adolescents. This study aims to generate a preference-based scoring algorithm for the WAItE.

METHODS AND ANALYSIS

A discrete choice experiment (DCE) will be administered to value health states described by the WAItE classification system. These health states will be presented to members of the adult general population of the UK via an online survey. A range of regression models will be used to produce the utility algorithm for the WAItE. The DCE-visual analogue scale and time trade-off (TTO) anchoring methods will be used anchor the value set on to the 0-1 QALY scale.

ETHICS AND DISSEMINATION

The Newcastle University Medical School Ethics Committee approved the study (references 4772/2020 (DCE) and 9978/2020 (TTO)). The developed algorithm can be applied to future economic evaluations of weight management interventions and treatments for adolescents.

Eliciting gastric cancer survivors' preferences for follow-up services: a discrete choice experiment protocol

INTRODUCTION

Follow-up care is important for gastric cancer survivors, but follow-up strategies for gastric cancer survivors remain inconsistent, and compliance of gastric cancer survivors with follow-up care is very low. Understanding the needs and preferences of gastric cancer survivors is conducive to developing appropriate and acceptable follow-up strategies, thereby improving patient compliance. Discrete choice experiments can quantify individual needs and preferences. However, to date, there is no discrete choice experiment on the preferences of gastric cancer survivors, and no studies have examined how gastric cancer survivors make choices based on different characteristics of follow-up. This paper outlines an ongoing discrete choice experiment that aims to (1) explore follow-up service-related characteristics that may affect gastric cancer survivors' choices about their follow-up, (2) elicit how gastric cancer survivors consider the trade-offs among different follow-up service options using discrete choice experiment, (3) determine whether gastric cancer survivors' needs and preferences for follow-up vary due to the economy, politics, technology and culture in different regions.

METHODS AND ANALYSIS

Six attributes were developed through a literature review, semistructured interviews and experts and focus group discussions. A fractional factorial design was used to evaluate the interaction between attributes. A multiple logit model will be used to understand the trade-off between the follow-up characteristics of gastric cancer survivors. A mixed logit model will be used to explore the willingness to pay and uptake rate of gastric cancer survivors for follow-up attributes and further explore the preferences of different groups.

ETHICS AND DISSEMINATION

This study was approved by the ethics committee of the School of Nursing, Jilin University. The results of this study will be shared through online blogs, policy briefs, seminars and peer-reviewed journal articles and will be used to modify the current strategy of gastric cancer survivors' follow-up services according to economic development and regional culture.

Discrete Choice Experiments to Elicit Patient Preferences for Decision Making in Transplantation

Providing care for transplant recipients is challenging given the need to maintain optimal graft function and survival while managing the debilitating side effects and complications associated with immunosuppression including infection, cancer, new-onset diabetes mellitus, and cardiovascular disease. Given the complexity of treatment options and the uncertainty about long-term benefits and harms of treatment, understanding patient preferences and values are key to ensuring that clinical decisions take into consideration patient priorities to support shared decision making and self-management. Choice experiments are increasingly used to quantify patient and community preferences, including in the field of transplantation. Discrete choice experiments (DCEs) are a well-established, validated methodology used to elicit preferences for decision making in health and other settings. In transplantation, for example, DCEs have been used to elicit patient preferences for outcomes following kidney transplantation, to identify community preferences factors for organ allocation and in establishing core outcomes. This article provides an overview of the concepts and methods used in the design of DCEs and how patients' preferences can be applied in shared decision making in transplantation.

Attribute Selection for a Discrete Choice Experiment Incorporating a Best-Worst Scaling Survey

OBJECTIVES

Although literature exists on using qualitative methods to generate potential attributes for a discrete choice experiment (DCE), there is little on selecting which attributes to include. We present a case study in which a best-worst scaling case 1 (BWS-1) survey was used to guide attribute selection for a DCE. The case study's context was the decision making of professionals around the choice of augmentative and alternative communication (AAC) systems for children with limited natural speech.

METHODS

BWS-1 survey attributes were generated from literature reviews and focus groups. DCE attributes were selected from BWS-1 attributes. The selection criteria were: include mostly important attributes; create coherent descriptions of children and AAC systems; address the project's research aims; have an appropriate respondent burden. Attributes' importance was judged using BWS-1 relative importance scores.

RESULTS

The BWS-1 survey included 19 child and 18 AAC device/system attributes and was administered to N = 93 AAC professionals. Four child and five device/system attributes were selected for the DCE, administered to N = 155 AAC professionals.

CONCLUSIONS

In this case study BWS-1 results were useful in DCE attribute selection. Four recommendations are made for future studies: define selection criteria for DCE attributes a priori; consider the impact participant's perspective will have on BWS-1 and DCE results; clearly define key terminology at the start of the study and refine it as the study progresses to reflect interim findings; BWS will be useful when there is little existing stated preference work on a topic and/or qualitative work is difficult.

Discrete choice experiments or best-worst scaling? A qualitative study to determine the suitability of preference elicitation tasks in research with children and young people

BACKGROUND

Ordinal tasks are increasingly used to explore preferences for health states. This study aimed to determine the suitability of two ordinal preference elicitation tasks (discrete choice experiments (DCE) and best-worst scaling (BWS)) for use with children and young people to generate health state utility values. The study explored children's understanding, the relationship between their age and level of understanding, and how many tasks they felt they could complete.

METHODS

Children aged 11-16 years were recruited from a secondary school in South Yorkshire, UK. Participants were asked to 'think aloud' as they completed a computer-based survey that contained both DCE and BWS tasks relating to dental caries (tooth decay) health states. Health states involved descriptions of the impact of tooth decay on children's daily lives. One-to-one semi-structured interviews were then held with participants, with use of a topic guide. Qualitative data were transcribed verbatim and analysed thematically.

RESULTS

A total of 33 children (12 male, 21 female) participated, comprising 5-6 children from each school year group. Children expressed a preference for BWS and demonstrated a better understanding of these tasks than DCE. There was no clear relationship between children's level of understanding and age. Children felt they could manage between 8 and 10 BWS tasks comfortably.

CONCLUSION

This study suggests that BWS tasks are the most appropriate type of preference elicitation task to value health states for children and young people aged 11-16 years to complete.

French Value-Set of the QLU-C10D, a Cancer-Specific Utility Measure Derived from the QLQ-C30

BACKGROUND AND OBJECTIVE

The EORTC Quality of Life Utility Measure-Core 10 Dimensions (QLU-C10D) is a new multi-attribute utility instrument derived from the EORTC Quality of Life Questionnaire-Core 30 (QLQ-C30), a widely used cancer-specific quality-of-life questionnaire. It covers ten dimensions: physical, role functioning, social, emotional functioning, pain, fatigue, sleep, appetite, nausea and bowel problems. To allow national health preferences to be reflected, country-specific valuations are being performed through collaboration between the Multi-Attribute Utility Cancer (MAUCa) Consortium and the EORTC. The aim of this study was to determine the utility weights for health states in the French version of the QLU-C10D.

METHODS

Valuations were run in a web-based setting in a general population sample of 1033 adults. Utilities were elicited using a discrete-choice experiment (DCE). Data were analyzed by conditional logistic regression and mixed logits.

RESULTS

The sample was representative of the general French population in terms of gender and age. Dimensions with the largest impact on utility weights were, in this order: physical functioning, pain and emotional functioning. The impact on utilities was lower for role functioning, nausea, bowel problems and social functioning. The dimensions of sleep, fatigue and lacking appetite were associated with the smallest utility decrement.

CONCLUSION

The results of the present study provide utility weights for the QLU-C10D and offer interesting prospects, as some cancer-specific dimensions also received sizeable utility weights (nausea and bowel problems). In fact, the EQ-5D and the HUI 3 are recommended in France and commonly used for cancer-related CUA; however, both these instruments are generic. The availability of a new cancer-specific utility instrument, such as the QLU-C10D, could improve the quality and the pertinence of future CUA in oncology.

Opening the 'Black Box': An Overview of Methods to Investigate the Decision-Making Process in Choice-Based Surveys

The desire to understand the preferences of patients, healthcare professionals and the public continues to grow. Health valuation studies, often in the form of discrete choice experiments, a choice based survey approach, proliferate as a result. A variety of methods of pre-choice process analysis have been developed to investigate how and why people make their decisions in such experiments and surveys. These techniques have been developed to investigate how people acquire and process information and make choices. These techniques offer the potential to test and improve theories of choice and/or associated empirical models. This paper provides an overview of such methods, with the focus on their use in stated choice-based healthcare studies. The methods reviewed are eye tracking, mouse tracing, brain imaging, deliberation time analysis and think aloud. For each method, we summarise the rationale, implementation, type of results generated and associated challenges, along with a discussion of possible future developments.

What Matters 2 Adults: a study protocol to develop a new preference-based wellbeing measure with Aboriginal and Torres Strait Islander adults (WM2Adults)

BACKGROUND

Understandings of health and wellbeing are culturally bound. Many Aboriginal and Torres Strait Islander people perceive wellbeing and quality of life (QOL) differently from the Western biomedical models of health underpinning existing QOL instruments. Any instrument to measure the wellbeing of Aboriginal and Torres Strait Islander people should be culturally appropriate and safe, include relevant dimensions, and be informed by their own values and preferences. Existing QOL instruments do not meet these standards. This study will generate a new preference-based wellbeing measure, WM2Adults, for Aboriginal and Torres Strait Islander adults, underpinned by their values and preferences.

METHODS

A mixed methods approach will be used; we will employ decolonising methodologies, privilege Aboriginal and Torres Strait Islander voices and perspectives, and adopt a strengths-based approach rather than a deficit lens. Yarning Circles will be conducted with Aboriginal and Torres Strait Islander people across Australia. A candidate item pool will be developed from these data, on which psychometric analysis and validity testing will be undertaken to develop a descriptive system. Following finalisation of the descriptive system, wellbeing states will be valued using a quantitative preference-based approach (best-worst scaling) with a diverse sample of Aboriginal and Torres Strait Islander adults (n = 1000). A multinomial (conditional) logit framework will be used to analyse responses and generate a scoring algorithm for the new preference-based WM2Adults measure.

DISCUSSION

The new wellbeing measure will have wide applicability in assessing the effectiveness and cost-effectiveness of new programs and services for Aboriginal and Torres Strait Islander people. Results will be disseminated through journals, conferences and policy forums, and will be shared with Aboriginal and Torres Strait Islander communities, organisations and research participants.

Patient Preferences in Surveillance: Findings From a Discrete Choice Experiment in the "My Follow-Up" Study

OBJECTIVES

Approximately 800 000 people die globally from colorectal cancer (CRC) every year. Prevention programs promote early detection, but for people with precancerous lesions, tailoring surveillance to include lifestyle-change programs could enhance prevention potential and improve outcomes.

METHODS

Those with intermediate or high-risk polyps removed during CRC screening colonoscopy within the Northern Ireland CRC Screening Programme were invited to complete 8 discrete choice questions about tailored surveillance, analyzed using random-parameters logit and a latent class modeling approach.

RESULTS

A total of 231 participants (77% male) self-reported comorbid hypertension (53%), high cholesterol (48%), and mean body mass index of 28.7 (overweight). Although 39% of participants were unaware of their CRC risk status, 30.9% indicated they were already making changes to reduce their risk. Although all respondents were significantly risk- and cost-averse, the latent class analysis identified 3 segments (classes): 1. Class 1 (26.8%) significantly favored phone or email support for a lifestyle change, a 17-month testing interval, and noninvasive testing. 2. Class 2 (48.4%) preferred the status quo. 3. Class 3 (24.7%) significantly favored further risk reduction and invasive testing.

CONCLUSIONS

This is the first documented preference study focusing on postpolypectomy surveillance offering lifestyle interventions. Although current care is strongly preferred, risk and cost aversion are important for participants. Latent class analysis shows that some respondents are willing to change diet and lifestyle behaviors, reflecting a teachable moment, with opportunities to personalize and optimize surveillance. Significant discordance between perceived and known risk of recurrence and limited recall of risk information provided within current practice suggest necessary improvements to surveillance programs.

iSAY (incentives for South African youth): Stated preferences of young people living with HIV

High adherence to antiretroviral therapy (ART) is essential for achieving viral suppression and preventing HIV transmission. Yet adherence is suboptimal among adolescents who face unique adherence challenges. Little is known about the role of conditional economic incentives (CEIs) for increasing ART adherence in this population. During 2017-2019, we conducted a mixed-methods discrete choice experiment in Cape Town, South Africa to inform the optimal design of a CEI intervention for ART adherence among youth. In-depth interviews were conducted with n = 35 adolescents (10-19 years old) living with HIV and prescribed ART, to identify attributes of a youth-centered CEI intervention for ART adherence. A discrete choice experiment was subsequently conducted with N = 168 adolescents to elicit preferences for intervention components. A rank-ordered mixed logit model was used for main results; marginal willingness-to-accept (mWTA) was then estimated. Five attributes emerged from the qualitative research as important for a CEI-based intervention for youth ART adherence: (1) incentive amount, (2) incentive format, (3) incentive recipient, (4) delivery mode, and (5) program participants. Youth had a high probability of acceptance of any incentives program (88-100%), yet they did not have a strong preference of a quarterly over a monthly program. From a maximum incentive amount of R1920 (~US$115), youth were willing to forgo up to R126 per year (~US$9) if the incentive was given in cash (versus fashion vouchers); R274 (~US$19.6) if it was open to both previously adherent and non-adherent youth (instead of non-adherent only); and up to R91 (~US$6.5) to receive incentives at a clinic setting (instead of electronically). The use of incentives over the short term during the critical age- and developmental-transition, when adolescents begin to take sole responsibility for their medication-taking behaviors, holds great promise for habituating adherence into adulthood.

How do people with dementia and family carers value dementia-specific quality of life states? An explorative "Think Aloud" study

OBJECTIVE

To investigate the decision-making processes applied by people with dementia and family carers participating in using health economic approaches to value dementia-specific quality of life states.

METHODS

People with dementia (n = 13) and family carers (n = 14) participated in valuing quality of life states using two health economic approaches: Discrete Choice Experiment (DCE) and Best Worst Scaling (BWS). Participants were encouraged to explain their reasoning using a "Think Aloud" approach.

RESULTS

People with dementia and family carers adopted a range of decision-making strategies including "anchoring" the presented states against current quality of life, or simplifying the decision-making by focusing on the sub-set of attributes deemed most important. Overall, there was strong evidence of task engagement for BWS and DCE.

CONCLUSIONS

Health economic valuation approaches can be successfully applied with people with dementia and family carers. These data can inform the assessment of benefits from their perspectives for incorporation within economic evaluation.

Carer Social Care-Related Quality of Life Outcomes: Estimating English Preference Weights for the Adult Social Care Outcomes Toolkit for Carers

BACKGROUND

There is increasing interest in assessing the effects of interventions on older people, people with long-term conditions and their informal carers for use in economic evaluation. The Adult Social Care Outcomes Toolkit for Carers (ASCOT-Carer) is a measure that specifically assesses the impact of social care services on informal carers. To date, the ASCOT-Carer has not been preference-weighted.

OBJECTIVES

To estimate preference-based index values for the English version of the ASCOT-Carer from the general population in England.

METHODS

The ASCOT-Carer consists of 7 domains, each reflecting aspects of social care-related quality of life in informal carers. Preferences for the ASCOT-Carer social care-related quality of life states were estimated using a best-worst scaling exercise in an online survey. The survey was administered to a sample of the general adult population in England (n = 1000). Participants were asked to put themselves into the hypothetical state of being an informal carer and indicate which attribute they thought was the best (first and second) and worst (first and second) from a profile list of 7 attributes reflecting the 7 domains, each ranging at a different level (1-4). Multinomial logit regression was used to analyze the data and estimate preference weights for the ASCOT-Carer measure.

RESULTS

The most valued aspect by English participants was the 'occupation' attribute at its highest level. Results further showed participants rated having no control over their daily life as the lowest attribute-level of all those presented. The position of the 7 attributes influenced participants' best and worst choices, and there was evidence of both scale and taste heterogeneity on preferences.

CONCLUSION

This study has established a set of preference-based index values for the ASCOT-Carer in England derived from the best-worst scaling exercise that can be used for economic evaluation of interventions on older individuals and their informal carers.

Application of a Discrete Choice Experiment to Assess Adherence-Related Motivation Among Adolescents and Young Adults With Cancer

Objective

To develop and pilot test a discrete choice experiment (DCE) to assess the specific factors likely to motivate a given adolescent or young adult with cancer to adhere to medications included in treatment protocols.

Methods

A multimethod and iterative approach was used. Twenty adolescents and young adults with cancer participated in DCE pilot testing and refinement. Hierarchical Bayes was used to estimate attribute and attribute-level preference scores for each participant.

Results

The final DCE included nine choice sets composed of four attributes. The resulting DCE was modified with feedback from 20 adolescents and young adults (M[SD] age = 18.66 [2.95]) with cancer. The DCE was understandable and relevant. The factors likely to motivate adolescents and young adults with cancer to be adherent differed across individuals.

Conclusions

DCEs including the measure developed in this manuscript offer a novel approach to understanding individual-level differences critical for informing adherence-promotion efforts.

Assessing quality of care in nursing homes using discrete choice experiments: How does the level of cognitive functioning impact upon older people's preferences?

Traditionally older people with mild to moderate cognitive impairment have been excluded from preference elicitation studies in health economics. We assessed the impact of the level of cognitive functioning on preference and scale heterogeneity in a discrete choice experiment undertaken with 126 older people living in residential aged care homes in Australia between January 2015 and February 2016. Data was analysed using conditional logit models for sub-groups of participants with mild to moderate cognitive impairment (N = 52) and without cognitive impairment (N = 74), and for the entire study sample using a heteroscedastic conditional logit regression model allowing for scale heterogeneity. The Swait-Louviere test was undertaken to formally test for differences in preference and scale between the two groups. Cognitive impairment was not significant in the scale function of the heteroscedastic conditional logit model (beta = -0.403, SE=0.341, p = 0.237). There were no statistical differences in estimated vector of preference parameters based on the presence or absence of cognitive impairment (Chi-squared = 13, 25 df, p = 0.976). Although there was evidence of a small increase in response variability with increasing cognitive impairment this did not reach statistical significance, and we were able to combine responses for people with and without cognitive impairment. Overall, the findings provide support for the more widespread inclusion of older people with mild to moderate cognitive impairment in such studies.

An Exploratory Application of Eye-Tracking Methods in a Discrete Choice Experiment

Background. Discrete choice experiments (DCEs) are increasingly used to elicit preferences for benefit-risk tradeoffs. The primary aim of this study was to explore how eye-tracking methods can be used to understand DCE respondents’ decision-making strategies. A secondary aim was to explore if the presentation and communication of risk affected respondents’ choices. Method. Two versions of a DCE were designed to understand the preferences of female members of the public for breast screening that varied in how risk attributes were presented. Risk was communicated as either 1) percentages or 2) icon arrays and percentages. Eye-tracking equipment recorded eye movements 1000 times a second. A debriefing survey collected sociodemographics and self-reported attribute nonattendance (ANA) data. A heteroskedastic conditional logit model analyzed DCE data. Eye-tracking data on pupil size, direction of motion, and total visual attention (dwell time) to predefined areas of interest were analyzed using ordinary least squares regressions. Results. Forty women completed the DCE with eye-tracking. There was no statistically significant difference in attention (fixations) to attributes between the risk communication formats. Respondents completing either version of the DCE with the alternatives presented in columns made more horizontal (left-right) saccades than vertical (up-down). Eye-tracking data confirmed self-reported ANA to the risk attributes with a 40% reduction in mean dwell time to the “probability of detecting a cancer” (P = 0.001) and a 25% reduction to the “risk of unnecessary follow-up” (P = 0.008). Conclusion. This study is one of the first to show how eye-tracking can be used to understand responses to a health care DCE and highlighted the potential impact of risk communication on respondents’ decision-making strategies. The results suggested self-reported ANA to cost attributes may not be reliable.

"I Was Trying to Do the Maths": Exploring the Impact of Risk Communication in Discrete Choice Experiments

BACKGROUND

Risk is increasingly used as an attribute in discrete choice experiments (DCEs). However, risk and probabilities are complex concepts that can be open to misinterpretation, potentially undermining the robustness of DCEs as a valuation method. This study aimed to understand how respondents made benefit-risk trade-offs in a DCE and if these were affected by the communication of the risk attributes.

METHODS

Female members of the public were recruited via local advertisements to participate in think-aloud interviews when completing a DCE eliciting their preferences for a hypothetical breast screening programme described by three attributes: probability of detecting a cancer; risk of unnecessary follow-up; and cost of screening. Women were randomised to receive risk information as either (1) percentages or (2) percentages and icon arrays. Interviews were digitally recorded then transcribed to generate qualitative data for thematic analysis.

RESULTS

Nineteen women completed the interviews (icon arrays n = 9; percentages n = 10). Analysis revealed four key themes where women made references to (1) the nature of the task; (2) their feelings; (3) their experiences, for instance making analogies to similar risks; and (4) economic phenomena such as opportunity costs and discounting.

CONCLUSION

Most women completed the DCE in line with economic theory; however, violations were identified. Women appeared to visualise risk whether they received icon arrays or percentages only. Providing clear instructions and graphics to aid interpretation of risk and qualitative piloting to verify understanding is recommended. Further investigation is required to determine if the process of verbalising thoughts changes the behaviour of respondents.

A Systematic Review Comparing the Acceptability, Validity and Concordance of Discrete Choice Experiments and Best-Worst Scaling for Eliciting Preferences in Healthcare

OBJECTIVE

The aim of this study was to compare the acceptability, validity and concordance of discrete choice experiment (DCE) and best-worst scaling (BWS) stated preference approaches in health.

METHODS

A systematic search of EMBASE, Medline, AMED, PubMed, CINAHL, Cochrane Library and EconLit databases was undertaken in October to December 2016 without date restriction. Studies were included if they were published in English, presented empirical data related to the administration or findings of traditional format DCE and object-, profile- or multiprofile-case BWS, and were related to health. Study quality was assessed using the PREFS checklist.

RESULTS

Fourteen articles describing 12 studies were included, comparing DCE with profile-case BWS (9 studies), DCE and multiprofile-case BWS (1 study), and profile- and multiprofile-case BWS (2 studies). Although limited and inconsistent, the balance of evidence suggests that preferences derived from DCE and profile-case BWS may not be concordant, regardless of the decision context. Preferences estimated from DCE and multiprofile-case BWS may be concordant (single study). Profile- and multiprofile-case BWS appear more statistically efficient than DCE, but no evidence is available to suggest they have a greater response efficiency. Little evidence suggests superior validity for one format over another. Participant acceptability may favour DCE, which had a lower self-reported task difficulty and was preferred over profile-case BWS in a priority setting but not necessarily in other decision contexts.

CONCLUSION

DCE and profile-case BWS may be of equal validity but give different preference estimates regardless of the health context; thus, they may be measuring different constructs. Therefore, choice between methods is likely to be based on normative considerations related to coherence with theoretical frameworks and on pragmatic considerations related to ease of data collection.

Relevance of barriers and facilitators in the use of health technology assessment in Colombia

OBJECTIVES

Several studies, mostly from developed countries, have identified barriers and facilitators with regard to the uptake of health technology assessment (HTA). This study elicited, using best-worst scaling (BWS), what HTA experts in Colombia consider to be the most important barriers and facilitators in the use of HTA, and makes a comparison to results from the Netherlands.

METHODS

Two object case surveys (one for barriers, one for facilitators) were conducted among 18 experts (policymakers, health professionals, PhD students, senior HTA-researchers) from Colombia. Seven respondents were employees of the national HTA agency Instituto de Evaluación Tecnológica de Salud (IETS). In total, 22 barriers and 19 facilitators were included. In each choice task, participants were asked to choose the most and least important barrier/facilitator from a set of five. Hierarchical Bayes modeling was used to compute the mean relative importance scores (RIS) for each factor, and a subgroup analysis was conducted to assess differences between IETS and non-IETS respondents. The final ranking was further compared to the results from a similar study conducted in the Netherlands.

RESULTS

The three most important barriers (RIS >6.00) were "Inadequate presentation format", "Absence of policy networks", and "Insufficient legal support". The six most important facilitators (RIS >6.00) were "Appropriate timing", "Clear presentation format", "Improving longstanding relation", "Appropriate incentives", "Sufficient qualified human resources", and "Availability to relevant HTA research". The perceived relevance of the barriers and facilitators differed slightly between IETS and non-IETS employees, while the differences between the rankings in Colombia and the Netherlands were substantial.

CONCLUSION

The study suggests that barriers and facilitators related to technical aspects of processing HTA reports and to the contact and interaction between researchers and policymakers had the greatest importance in Colombia.

Developing a dementia-specific preference--based quality of life measure (AD-5D) in Australia: a valuation study protocol

INTRODUCTION

Generic instruments for assessing health-related quality of life may lack the sensitivity to detect changes in health specific to certain conditions, such as dementia. The Quality of Life in Alzheimer's Disease (QOL-AD) is a widely used and well-validated condition-specific instrument for assessing health-related quality of life for people living with dementia, but it does not enable the calculation of quality-adjusted life years, the basis of cost utility analysis. This study will generate a preference-based scoring algorithm for a health state classification system -the Alzheimer's Disease Five Dimensions (AD-5D) derived from the QOL-AD.

METHODS AND ANALYSIS

Discrete choice experiments with duration (DCE_TTO) and best-worst scaling health state valuation tasks will be administered to a representative sample of 2000 members of the Australian general population via an online survey and to 250 dementia dyads (250 people with dementia and their carers) via face-to-face interview. A multinomial (conditional) logistic framework will be used to analyse responses and produce the utility algorithm for the AD-5D.

ETHICS AND DISSEMINATION

The algorithms developed will enable prospective and retrospective economic evaluation of any treatment or intervention targeting people with dementia where the QOL-AD has been administered and will be available online. Results will be disseminated through journals that publish health economics articles and through professional conferences. This study has ethical approval.

Using Discrete Choice Experiments to Inform the Benefit-Risk Assessment of Medicines: Are We Ready Yet?

There is emerging interest in the use of discrete choice experiments as a means of quantifying the perceived balance between benefits and risks (quantitative benefit-risk assessment) of new healthcare interventions, such as medicines, under assessment by regulatory agencies. For stated preference data on benefit-risk assessment to be used in regulatory decision making, the methods to generate these data must be valid, reliable and capable of producing meaningful estimates understood by decision makers. Some reporting guidelines exist for discrete choice experiments, and for related methods such as conjoint analysis. However, existing guidelines focus on reporting standards, are general in focus and do not consider the requirements for using discrete choice experiments specifically for quantifying benefit-risk assessments in the context of regulatory decision making. This opinion piece outlines the current state of play in using discrete choice experiments for benefit-risk assessment and proposes key areas needing to be addressed to demonstrate that discrete choice experiments are an appropriate and valid stated preference elicitation method in this context. Methodological research is required to establish: how robust the results of discrete choice experiments are to formats and methods of risk communication; how information in the discrete choice experiment can be presented effectually to respondents; whose preferences should be elicited; the correct underlying utility function and analytical model; the impact of heterogeneity in preferences; and the generalisability of the results. We believe these methodological issues should be addressed, alongside developing a 'reference case', before agencies can safely and confidently use discrete choice experiments for quantitative benefit-risk assessment in the context of regulatory decision making for new medicines and healthcare products.

Comparing Analytic Hierarchy Process and Discrete-Choice Experiment to Elicit Patient Preferences for Treatment Characteristics in Age-Related Macular Degeneration

BACKGROUND

In this study, we conducted an analytic hierarchy process (AHP) and a discrete choice experiment (DCE) to elicit the preferences of patients with age-related macular degeneration using identical attributes and levels.

OBJECTIVES

To compare preference-based weights for age-related macular degeneration treatment attributes and levels generated by two elicitation methods. The properties of both methods were assessed, including ease of instrument use.

METHODS

A DCE and an AHP experiment were designed on the basis of five attributes. Preference-based weights were generated using the matrix multiplication method for attributes and levels in AHP and a mixed multinomial logit model for levels in the DCE. Attribute importance was further compared using coefficient (DCE) and weight (AHP) level ranges. The questionnaire difficulty was rated on a qualitative scale. Patients were asked to think aloud while providing their judgments.

RESULTS

AHP and DCE generated similar results regarding levels, stressing a preference for visual improvement, frequent monitoring, on-demand and less frequent injection schemes, approved drugs, and mild side effects. Attribute weights derived on the basis of level ranges led to a ranking that was opposite to the AHP directly calculated attribute weights. For example, visual function ranked first in the AHP and last on the basis of level ranges.

CONCLUSIONS

The results across the methods were similar, with one exception: the directly measured AHP attribute weights were different from the level-based interpretation of attribute importance in both DCE and AHP. The dependence/independence of attribute importance on level ranges in DCE and AHP, respectively, should be taken into account when choosing a method to support decision making.

Exploring expert opinion on the practicality and effectiveness of biosecurity measures on dairy farms in the United Kingdom using choice modeling

Biosecurity, defined as a series of measures aiming to stop disease-causing agents entering or leaving an area where farm animals are present, is very important for the continuing economic viability of the United Kingdom dairy sector, and for animal welfare. This study gathered expert opinion from farmers, veterinarians, consultants, academics, and government and industry representatives on the practicality and effectiveness of different biosecurity measures on dairy farms. The study used best-worst scaling, a technique that allows for greater discrimination between choices and avoids the variability in interpretation associated with other methods, such as Likert scales and ranking methods. Keeping a closed herd was rated as the most effective measure overall, and maintaining regular contact with the veterinarian was the most practical measure. Measures relating to knowledge, planning, and veterinary involvement; buying-in practices; and quarantine and treatment scored highly for effectiveness overall. Measures relating to visitors, equipment, pest control, and hygiene scored much lower for effectiveness. Overall, measures relating to direct animal-to-animal contact scored much higher for effectiveness than measures relating to indirect disease transmission. Some of the most effective measures were also rated as the least practical, such as keeping a closed herd and avoiding nose-to-nose contact between contiguous animals, suggesting that real barriers exist for farmers when implementing biosecurity measures on dairy farms. We observed heterogeneity in expert opinion on biosecurity measures; for example, veterinarians rated the effectiveness of consulting the veterinarian on biosecurity significantly more highly than dairy farmers, suggesting a greater need for veterinarians to promote their services on-farm. Still, both groups rated it as a practical measure, suggesting that the farmer-veterinarian relationship holds some advantages for the promotion of biosecurity.

Experimental measurement of preferences in health and healthcare using best-worst scaling: an overview

Best-worst scaling (BWS), also known as maximum-difference scaling, is a multiattribute approach to measuring preferences. BWS aims at the analysis of preferences regarding a set of attributes, their levels or alternatives. It is a stated-preference method based on the assumption that respondents are capable of making judgments regarding the best and the worst (or the most and least important, respectively) out of three or more elements of a choice-set. As is true of discrete choice experiments (DCE) generally, BWS avoids the known weaknesses of rating and ranking scales while holding the promise of generating additional information by making respondents choose twice, namely the best as well as the worst criteria. A systematic literature review found 53 BWS applications in health and healthcare. This article expounds possibilities of application, the underlying theoretical concepts and the implementation of BWS in its three variants: 'object case', 'profile case', 'multiprofile case'. This paper contains a survey of BWS methods and revolves around study design, experimental design, and data analysis. Moreover the article discusses the strengths and weaknesses of the three types of BWS distinguished and offered an outlook. A companion paper focuses on special issues of theory and statistical inference confronting BWS in preference measurement.

Using Best-Worst Scaling to Investigate Preferences in Health Care

INTRODUCTION

Best-worst scaling (BWS) is becoming increasingly popular to elicit preferences in health care. However, little is known about current practice and trends in the use of BWS in health care. This study aimed to identify, review and critically appraise BWS in health care, and to identify trends over time in key aspects of BWS.

METHODS

A systematic review was conducted, using Medline (via Pubmed) and EMBASE to identify all English-language BWS studies published up until April 2016. Using a predefined extraction form, two reviewers independently selected articles and critically appraised the study quality, using the Purpose, Respondents, Explanation, Findings, Significance (PREFS) checklist. Trends over time periods (≤2010, 2011, 2012, 2013, 2014 and 2015) were assessed further.

RESULTS

A total of 62 BWS studies were identified, of which 26 were BWS object case studies, 29 were BWS profile case studies and seven were BWS multi-profile case studies. About two thirds of the studies were performed in the last 2 years. Decreasing sample sizes and decreasing numbers of factors in BWS object case studies, as well as use of less complicated analytical methods, were observed in recent studies. The quality of the BWS studies was generally acceptable according to the PREFS checklist, except that most studies did not indicate whether the responders were similar to the non-responders.

CONCLUSION

Use of BWS object case and BWS profile case has drastically increased in health care, especially in the last 2 years. In contrast with previous discrete-choice experiment reviews, there is increasing use of less sophisticated analytical methods.

Measuring patients' priorities using the Analytic Hierarchy Process in comparison with Best-Worst-Scaling and rating cards: methodological aspects and ranking tasks

BACKGROUND

Identifying patient priorities and preference measurements have gained importance as patients claim a more active role in health care decision making. Due to the variety of existing methods, it is challenging to define an appropriate method for each decision problem. This study demonstrates the impact of the non-standardized Analytic Hierarchy Process (AHP) method on priorities, and compares it with Best-Worst-Scaling (BWS) and ranking card methods.

METHODS

We investigated AHP results for different Consistency Ratio (CR) thresholds, aggregation methods, and sensitivity analyses. We also compared criteria rankings of AHP with BWS and ranking cards results by Kendall's tau b.

RESULTS

The sample for our decision analysis consisted of 39 patients with rare diseases and mean age of 53.82 years. The mean weights of the two groups of CR ≤ 0.1 and CR ≤ 0.2 did not differ significantly. For the aggregation by individual priority (AIP) method, the CR was higher than for aggregation by individual judgment (AIJ). In contrast, the weights of AIJ were similar compared to AIP, but some criteria's rankings differed. Weights aggregated by geometric mean, median, and mean showed deviating results and rank reversals. Sensitivity analyses showed instable rankings. Moderate to high correlations between the rankings resulting from AHP and BWS.

LIMITATIONS

Limitations were the small sample size and the heterogeneity of the patients with different rare diseases.

CONCLUSION

In the AHP method, the number of included patients is associated with the threshold of the CR and choice of the aggregation method, whereas both directions of influence could be demonstrated. Therefore, it is important to implement standards for the AHP method. The choice of method should depend on the trade-off between the burden for participants and possibilities for analyses.

Effectiveness and acceptability of parental financial incentives and quasi-mandatory schemes for increasing uptake of vaccinations in preschool children: systematic review, qualitative study and discrete choice experiment

BACKGROUND

Uptake of preschool vaccinations is less than optimal. Financial incentives and quasi-mandatory policies (restricting access to child care or educational settings to fully vaccinated children) have been used to increase uptake internationally, but not in the UK.

OBJECTIVE

To provide evidence on the effectiveness, acceptability and economic costs and consequences of parental financial incentives and quasi-mandatory schemes for increasing the uptake of preschool vaccinations.

DESIGN

Systematic review, qualitative study and discrete choice experiment (DCE) with questionnaire.

SETTING

Community, health and education settings in England.

PARTICIPANTS

Qualitative study - parents and carers of preschool children, health and educational professionals. DCE - parents and carers of preschool children identified as 'at high risk' and 'not at high risk' of incompletely vaccinating their children.

DATA SOURCES

Qualitative study - focus groups and individual interviews. DCE - online questionnaire.

REVIEW METHODS

The review included studies exploring the effectiveness, acceptability or economic costs and consequences of interventions that offered contingent rewards or penalties with real material value for preschool vaccinations, or quasi-mandatory schemes that restricted access to 'universal' services, compared with usual care or no intervention. Electronic database, reference and citation searches were conducted.

RESULTS

Systematic review - there was insufficient evidence to conclude that the interventions considered are effective. There was some evidence that the quasi-mandatory interventions were acceptable. There was insufficient evidence to draw conclusions on economic costs and consequences. Qualitative study - there was little appetite for parental financial incentives. Quasi-mandatory schemes were more acceptable. Optimising current services was consistently preferred to the interventions proposed. DCE and questionnaire - universal parental financial incentives were preferred to quasi-mandatory interventions, which were preferred to targeted incentives. Those reporting that they would need an incentive to vaccinate their children completely required around £110. Those who did not felt that the maximum acceptable incentive was around £70.

LIMITATIONS

Systematic review - a number of relevant studies were excluded as they did not meet the study design inclusion criteria. Qualitative study - few partially and non-vaccinating parents were recruited. DCE and questionnaire - data were from a convenience sample.

CONCLUSIONS

There is little current evidence on the effectiveness or economic costs and consequences of parental financial incentives and quasi-mandatory interventions for preschool vaccinations. Universal incentives are likely to be more acceptable than targeted ones. Preferences concerning incentives versus quasi-mandatory interventions may depend on the context in which these are elicited.

FUTURE WORK

Further evidence is required on (i) the effectiveness and optimal configuration of parental financial incentive and quasi-mandatory interventions for preschool vaccinations - if effectiveness is confirmed, further evidence is required on how to communicate this to stakeholders and the impact on acceptability; and (ii) the acceptability of parental financial incentive and quasi-mandatory interventions for preschool vaccinations to members of the population who are not parents of preschool children or relevant health professionals. Further consideration should be given to (i) incorporating reasons for non-vaccination into new interventions for promoting vaccination uptake; and (ii) how existing services can be optimised.

STUDY REGISTRATION

This study is registered as PROSPERO CRD42012003192.

FUNDING

The National Institute for Health Research Health Technology Assessment programme.

Older, vulnerable patient view: a pilot and feasibility study of the patient measure of safety (PMOS) with patients in Australia

OBJECTIVES

The UK-developed patient measure of safety (PMOS) is a validated tool which captures patient perceptions of safety in hospitals. We aimed (1) to investigate the extent to which the PMOS is appropriate for use with stroke, acute myocardial infarction (AMI) and hip fracture patients in Australian hospitals and (2) to pilot the PMOS for use in a large-scale, national study 'Deepening our Understanding of Quality in Australia' (DUQuA).

PARTICIPANTS

Stroke, AMI and hip fracture patients (n=34) receiving care in 3 wards in 1 large hospital.

METHODS

2 phases were conducted. First, a 'think aloud' study was used to determine the validity of PMOS with this population in an international setting, and to make amendments based on patient feedback. The second phase tested the revised measure to establish the internal consistency reliability of the revised subscales, and piloted the recruitment and administration processes to ensure feasibility of the PMOS for use in DUQuA.

RESULTS

Of the 43 questions in the PMOS, 13 (30%) were amended based on issues patients highlighted for improvement in phase 1. In phase 2, a total of 34 patients were approached and 29 included, with a mean age of 71.3 years (SD=16.39). Internal consistency reliability was established using interitem correlation and Cronbach's α for all but 1 subscale. The most and least favourably rated aspects of safety differed between the 3 wards. A study log was categorised into 10 key feasibility factors, including liaising with wards to understand operational procedures and identify patterns of patient discharge.

CONCLUSIONS

Capturing patient perceptions of care is crucial in improving patient safety. The revised PMOS is appropriate for use with vulnerable older adult groups. The findings from this study have informed key decisions made for the deployment of this measure as part of the DUQuA study.