Using Discrete Choice Experiments to Inform the Benefit-Risk Assessment of Medicines: Are We Ready Yet?

Understanding general practitioner and pharmacist preferences for pharmacogenetic testing in primary care: a discrete choice experiment

Pharmacogenetic testing in the United Kingdom's National Health Service (NHS) has historically been reactive in nature, undertaken in the context of single gene-drug relationships in specialist settings. Using a discrete choice experiment we aimed to identify healthcare professional preferences for development of a pharmacogenetic testing service in primary care in the NHS. Respondents, representing two professions groups (general practitioners or pharmacists), completed one of two survey versions, asking them to select their preferred pharmacogenetic testing service in the context of a presentation of low mood or joint pain. Responses from 235 individuals were included. All respondents preferred pharmacogenetic testing over no testing, though preference heterogeneity was identified. Both professional groups, but especially GPs, were highly sensitive to service design, with uptake varying depending on the service offered. This study demonstrates uptake of a pharmacogenetic testing service is impacted by service design and highlights key areas which should be prioritised within future initiatives.

The philosophy of pharmaceutical regulation-Paternalism or freedom of choice?

When assessing the value of new drugs regulatory authorities across the world frequently make different decisions even though their decisions are based on the same evidence package. In this perspective we argue that even in today's world regulatory and medical decision making is framed by conflicting philosophical schools of thought, namely the liberal tradition of the Anglo Saxon countries pioneered by the Scotsman Adam Smith and the continental European tradition of paternalism that roots back to the German philosopher Georg Friedrich Hegel. We outline the basics of these two philosophical theories and show that countries following the liberal tradition are more reluctant to reject new drugs due to weak evidence. Instead, they leave decisions to a greater extend to those who are affected, namely patients and their caregivers.

Patient and General Population Preferences Regarding the Benefits and Harms of Treatment for Metastatic Prostate Cancer: A Discrete Choice Experiment

Background

Patient preferences for treatment outcomes are important to guide decision-making in clinical practice, but little is known about the preferences of patients with metastatic hormone-sensitive prostate cancer (mHSPC).

Objective

To evaluate patient preferences regarding the attributed benefits and harms of systemic treatments for mHSPC and preference heterogeneity between individuals and specific subgroups.

Design setting and participants

We conducted an online discrete choice experiment (DCE) preference survey among 77 patients with metastatic prostate cancer (mPC) and 311 men from the general population in Switzerland between November 2021 and August 2022.

Outcome measurements and statistical analysis

We evaluated preferences and preference heterogeneity related to survival benefits and treatment-related adverse effects using mixed multinomial logit models and estimated the maximum survival time participants were willing to trade to avert specific adverse effects. We further assessed characteristics associated with different preference patterns via subgroup and latent class analyses.

Results and limitations

Patients with mPC showed an overall stronger preference for survival benefits in comparison to men from the general population (p = 0.004), with substantial preference heterogeneity between individuals within the two samples (both p < 0.001). There was no evidence of differences in preferences for men aged 45-65 yr versus ≥65 yr, patients with mPC in different disease stages or with different adverse effect experiences, or general population participants with and without experiences with cancer. Latent class analyses suggested the presence of two groups strongly preferring either survival or the absence of adverse effects, with no specific characteristic clearly associated with belonging to either group. Potential biases due to participant selection, cognitive burden, and hypothetical choice scenarios may limit the study results.

Conclusions

Given the relevant heterogeneity in participant preferences regarding the benefits and harms of treatment for mHSPC, patient preferences should be explicitly discussed during decision-making in clinical practice and reflected in clinical practice guidelines and regulatory assessment regarding treatment for mHSPC.

Patient summary

We examined the preferences (values and perceptions) of patients and men from the general population regarding the benefits and harms of treatment for metastatic prostate cancer. There were large differences between men in how they balanced the expected survival benefits and potential adverse effects. While some men strongly valued survival, others more strongly valued the absence of adverse effects. Therefore, it is important to discuss patient preferences in clinical practice.

Do preferences differ based on respondent experience of a health issue and its treatment? A case study using a public health intervention

OBJECTIVES

Preference information is increasingly being elicited to support decision-making. Although discrete choice experiments (DCEs) are commonly used, little is known about how respondents' relative experience of a health issue, and its treatment, might impact the results of preference studies. The aim of this study was to explore how preferences differ between groups of individuals with varying levels of experience of a health issue and its treatment, using a weight loss maintenance (WLM) programme as a case study.

METHODS

An online DCE survey was provided to four groups, each differing in their level of experience with weight loss and WLM programmes. One group was recruited from a randomised controlled trial of a WLM programme (ISRCTN14657176) and the other three from an online panel. Choice data were analysed using mixed logit models. Relative attribute importance scores and willingness-to-pay (WTP) estimates were estimated to enable comparisons between groups.

RESULTS

Preferences differed between the groups across different attributes. The largest differences related to the outcome (weight re-gain) and cost attributes, resulting in WTP estimates that were statistically significantly different. The most experienced group was willing to pay £0.35 (95% CI: £0.28, £0.42) to avoid a percentage point increase in weight re-gain, compared with £0.12 (95% CI: £0.08, £0.16) for the least experienced group.

CONCLUSION

This study provides evidence in a public health setting to suggest that preferences differ based on respondent experience of the health issue and its treatment. Health preference researchers should therefore carefully consider the appropriate composition of their study samples.

Patient preferences for epilepsy treatment: a systematic review of discrete choice experimental studies

BACKGROUND

This review aimed to 1) identify and assess the quality of discrete choice experiments (DCEs) examining preferences related to epilepsy treatment; 2) summarize the attributes and attribute levels measured in these studies; 3) identify how researchers selected and developed these attributes; and 4) identify which attributes are most important for epilepsy patients.

METHODS

A systematic literature review using PubMed, Web of Science and Scopus databases from database inception to February or April 2022. We included primary discrete-choice experiments eliciting preferences for various attributes of pharmacological and surgical interventions in patients diagnosed with epilepsy or the parents/carers of children with epilepsy. We excluded non- primary studies, studies assessing preferences for nonpharmacological treatment and studies that elicit preferences using methods other than discrete choice experiments. Two authors independently selected studies, extracted data and assessed risk of bias of studies. The quality of the included studies was assessed using two validated checklists. Study characteristics and findings were summarized descriptively.

RESULTS

A total of seven studies were included in the review. The majority of studies explored patients' preferences, and two compared the preferences of patients with physicians. The majority (n = 6) compared two medications, and one compared two surgical options to continuing medication options. The studies examined 44 attributes in total, including side effects (n = 26), efficacy expressed as being seizure free or have fewer seizures (n = 8), costs (n = 3), dosing frequency (n = 3), duration of side effects (n = 2), mortality (n = 1), long-term problems after surgery (n = 1) and surgical options (n = 1). The findings indicate that people with epilepsy have strong preferences for improving seizure control, which was ranked as the top priority in all studies. Patients also have a strong preference for the reduction of adverse effects and may be willing to make trade-offs between improved seizure control and reduction of long-term side effects that may impact their quality of life.

CONCLUSIONS

The use of DCEs in measuring patients' preference for epilepsy treatment is accumulating. However, inadequate reporting of methodological details may reduce decision-makers' confidence in the findings. Suggestions for future research are provided.

Preferences for support in managing symptoms of an asthma flare-up: a pilot study of a discrete choice experiment

Objective: Information on the preferences of people with asthma for support in managing a flare-up can inform service design which may facilitate appropriate help-seeking. To date, little is known about support preferences for managing a flare-up. The aim of this study was to develop and pilot a discrete choice experiment (DCE) to elicit the preferences of people with asthma with regards to support in managing a flare-up.Methods: Steps in developing the DCE included identification and selection of attributes and levels of the support services, construction of choice tasks, experimental design, construction of DCE instrument, and pretest (n=16) and pilot (n=38) studies of the DCE instrument. A multinomial logit model was used to examine the strength and direction of the six attributes in the pilot study.Results: Our results indicate that from a patient perspective, having a healthcare professional that listens to their concerns was the most valued attribute of support in asthma flare-up management. The other features of support valued by participants were timely access to consultation, a healthcare professional with knowledge of their patient history, a specialist doctor and face-to-face communication. Having a written action plan was the least valued attribute.Conclusions: Our findings suggest patient preference for a model of support in managing their symptoms which includes timely, face-to-face access to a healthcare professional that knows them and listens to their concerns. The findings of the pilot study need to be verified with a larger sample and using models to account for preference heterogeneity.

Does in vitro fertilization (IVF) treatment provide good value for money? A cost-benefit analysis

Background

Using traditional health technology assessment (HTA) outcome metrics, such as quality-adjusted life-years, to assess fertility treatments raises considerable methodological challenges because the objective of fertility treatments is to create new life rather than extend, save, or improve health-related quality of life.

Objective

The aim of this study was to develop a novel cost-benefit framework to assess value for money of publicly funded IVF treatment; to determine the number of cost-beneficial treatment cycles for women of different ages; and to perform an incremental cost-benefit analysis from a taxpayer perspective.

Methods

We developed a Markov model to determine the net monetary benefit (NMB) of IVF treatment by female age and number of cycles performed. IVF treatment outcomes were monetized using taxpayers' willingness-to-pay values derived from a discrete choice experiment (DCE). Using the current funding environment as the comparator, we performed an incremental analysis of only funding cost-beneficial cycles. Similar outputs to cost-effectiveness analyses were generated, including net-benefit acceptability curves and cost-benefit planes. We created an interactive online app to provide a detailed and transparent presentation of the results.

Results

The results suggest that at least five publicly funded IVF cycles are cost-beneficial in women aged <42 years. Cost-benefit planes suggest a strong taxpayer preference for restricting funding to cost-beneficial cycles over current funding arrangements in Australia from an economic perspective.

Conclusions

The provision of fertility treatment is valued highly by taxpayers. This novel cost-benefit method overcomes several challenges of conventional cost-effectiveness methods and provides an exemplar for incorporating DCE results into HTA. The results offer new evidence to inform discussions about treatment funding arrangements.

Accounting for Preference Heterogeneity in Discrete-Choice Experiments: An ISPOR Special Interest Group Report

OBJECTIVES

Discrete choice experiments (DCEs) are increasingly used to elicit preferences for health and healthcare. Although many applications assume preferences are homogenous, there is a growing portfolio of methods to understand both explained (because of observed factors) and unexplained (latent) heterogeneity. Nevertheless, the selection of analytical methods can be challenging and little guidance is available. This study aimed to determine the state of practice in accounting for preference heterogeneity in the analysis of health-related DCEs, including the views and experiences of health preference researchers and an overview of the tools that are commonly used to elicit preferences.

METHODS

An online survey was developed and distributed among health preference researchers and nonhealth method experts, and a systematic review of the DCE literature in health was undertaken to explore the analytical methods used and summarize trends.

RESULTS

Most respondents (n = 59 of 70, 84%) agreed that accounting for preference heterogeneity provides a richer understanding of the data. Nevertheless, there was disagreement on how to account for heterogeneity; most (n = 60, 85%) stated that more guidance was needed. Notably, the majority (n = 41, 58%) raised concern about the increasing complexity of analytical methods. Of the 342 studies included in the review, half (n = 175, 51%) used a mixed logit with continuous distributions for the parameters, and a third (n = 110, 32%) used a latent class model.

CONCLUSIONS

Although there is agreement about the importance of accounting for preference heterogeneity, there are noticeable disagreements and concerns about best practices, resulting in a clear need for further analytical guidance.

Systematic review of quantitative preference studies of treatments for rheumatoid arthritis among patients and at-risk populations

Treatments used for rheumatoid arthritis (RA) are under investigation for their efficacy to prevent RA in at risk groups. It is therefore important to understand treatment preferences of those at risk. We systematically reviewed quantitative preference studies of drugs to treat, or prevent RA, to inform the design of further studies and trials of RA prevention. Stated preference studies for RA treatment or prevention were identified through a search of five databases. Study characteristics and results were extracted, and the relative importance of different types of treatment attributes was compared across populations. Twenty three studies were included 20 of RA treatments (18 of patients; 2 of the general public) and 3 prevention studies with first-degree relatives (FDRs). Benefits, risks, administration method and cost (when included) were important determinants of treatment choice. A benefit was more important than a risk attribute in half of the studies of RA treatment that included a benefit attribute and 2/3 studies of RA prevention. There was variability in the relative importance of attributes across the few prevention studies. In studies with non-patient participants, attributes describing confidence in treatment effectiveness/safety were more important determinants of choice than in studies with patients. Most preference studies relating to RA are of treatments for established RA. Few studies examine preferences for treatments to prevent RA. Given intense research focus on RA prevention, additional preference studies in this context are needed. Variation in treatment preferences across different populations is not well understood and direct comparisons are needed.

A General Public Study on Preferences and Welfare Impacts of Antimicrobial Resistance in the United Kingdom

BACKGROUND

Antibiotics have led to considerable increases in life expectancy. However, over time, antimicrobial resistance has accelerated and is now a significant global public health concern. Understanding societal preferences for the use of antibiotics as well as eliciting the willingness to pay for future research is crucial.

OBJECTIVE

To investigate preferences for different strategies to optimize antibiotic use and to understand the willingness to pay for future research in antimicrobial resistance and antimicrobial drug development.

METHODS

A discrete-choice experiment was administered to a sample of the UK general population. Respondents were asked to make nine choices, each offering three options-two hypothetical "doctor and antibiotics" and one "no doctor-no antibiotics"-defined by five attributes: treatment, days needed to recover, risk of bacterial infection that needs antibiotics, risk of common side effects, and risk of antimicrobial resistance by 2050. Data were analyzed using random parameters logit models. A double-bounded contingent valuation was also included in the survey to explore the willingness to pay for policies to contain antimicrobial resistance.

RESULTS

Among the 2579 respondents who completed the survey, 1151 always selected "no doctor-no antibiotics" and 57 never varied their choices; therefore, 1371 responses were used in the analysis. Risk of antimicrobial resistance by 2050 was the most important attribute and the "treatment" was the least important attribute, although this was sensitive to a higher risk of bacterial infection. The aggregate annual willingness to pay for containing antimicrobial resistance was approximately £8.35 billion (~£5-£10 billion).

CONCLUSIONS

The antimicrobial resistance risk is relevant and important to the general public. The high willingness to pay suggests that large investments in policies or interventions to combat antimicrobial resistance are justified.

A Systematic Review of Discrete Choice Experiments in Oncology Treatments

BACKGROUND

As the number and type of cancer treatments available rises and patients live with the consequences of their disease and treatments for longer, understanding preferences for cancer care can help inform decisions about optimal treatment development, access, and care provision. Discrete choice experiments (DCEs) are commonly used as a tool to elicit stakeholder preferences; however, their implementation in oncology may be challenging if burdensome trade-offs (e.g. length of life versus quality of life) are involved and/or target populations are small.

OBJECTIVES

The aim of this review was to characterise DCEs relating to cancer treatments that were conducted between 1990 and March 2020.

DATA SOURCES

EMBASE, MEDLINE, and the Cochrane Database of Systematic Reviews were searched for relevant studies.

STUDY ELIGIBILITY CRITERIA

Studies were included if they implemented a DCE and reported outcomes of interest (i.e. quantitative outputs on participants' preferences for cancer treatments), but were excluded if they were not focused on pharmacological, radiological or surgical treatments (e.g. cancer screening or counselling services), were non-English, or were a secondary analysis of an included study.

ANALYSIS METHODS

Analysis followed a narrative synthesis, and quantitative data were summarised using descriptive statistics, including rankings of attribute importance.

RESULT

Seventy-nine studies were included in the review. The number of published DCEs relating to oncology grew over the review period. Studies were conducted in a range of indications (n = 19), most commonly breast (n =10, 13%) and prostate (n = 9, 11%) cancer, and most studies elicited preferences of patients (n = 59, 75%). Across reviewed studies, survival attributes were commonly ranked as most important, with overall survival (OS) and progression-free survival (PFS) ranked most important in 58% and 28% of models, respectively. Preferences varied between stakeholder groups, with patients and clinicians placing greater importance on survival outcomes, and general population samples valuing health-related quality of life (HRQoL). Despite the emphasis of guidelines on the importance of using qualitative research to inform attribute selection and DCE designs, reporting on instrument development was mixed.

LIMITATIONS

No formal assessment of bias was conducted, with the scope of the paper instead providing a descriptive characterisation. The review only included DCEs relating to cancer treatments, and no insight is provided into other health technologies such as cancer screening. Only DCEs were included.

CONCLUSIONS AND IMPLICATIONS

Although there was variation in attribute importance between responder types, survival attributes were consistently ranked as important by both patients and clinicians. Observed challenges included the risk of attribute dominance for survival outcomes, limited sample sizes in some indications, and a lack of reporting about instrument development processes.

PROTOCOL REGISTRATION

PROSPERO 2020 CRD42020184232.

Attributes Used for Cancer Screening Discrete Choice Experiments: A Systematic Review

BACKGROUND

Evidence from discrete choice experiments can be used to enrich understanding of preferences, inform the (re)design of screening programmes and/or improve communication within public campaigns about the benefits and harms of screening. However, reviews of screening discrete choice experiments highlight significant discrepancies between stated choices and real choices, particularly regarding willingness to undergo cancer screening. The identification and selection of attributes and associated levels is a fundamental component of designing a discrete choice experiment. Misspecification or misinterpretation of attributes may lead to non-compensatory behaviours, attribute non-attendance and responses that lack external validity.

OBJECTIVES

We aimed to synthesise evidence on attribute development, alongside an in-depth review of included attributes and methodological challenges, to provide a resource for researchers undertaking future studies in cancer screening.

METHODS

A systematic review was conducted to identify discrete choice experiments estimating preferences towards cancer screening, dated between 1990 and December 2020. Data were synthesised narratively. In-depth analysis of attributes led to classification into four categories: test specific, service delivery, outcomes and monetary. Attribute significance and relative importance were also analysed. The International Society for Pharmacoeconomics and Outcomes Research conjoint analysis checklist was used to assess the quality of reporting.

RESULTS

Forty-nine studies were included at full text. They covered a range of cancer sites: over half (26/49) examined colorectal screening. Most studies elicited general public preferences (34/49). In total, 280 attributes were included, 90% (252/280) of which were significant. Overall, test sensitivity and mortality reduction were most frequently found to be the most important to respondents.

CONCLUSIONS

Improvements in reporting the identification, selection and construction of attributes used within cancer screening discrete choice experiments are needed. This review also highlights the importance of considering the complexity of choice tasks when considering risk information or compound attributes. Patient and public involvement and stakeholder engagement are recommended to optimise understanding of unavoidably complex choice tasks throughout the design process. To ensure quality and maximise comparability across studies, further research is needed to develop a risk-of-bias measure for discrete choice experiments.

Patient preferences for stratified medicine in psoriasis: a discrete choice experiment

BACKGROUND

New technologies have enabled the potential for stratified medicine in psoriasis. It is important to understand patients' preferences to enable the informed introduction of stratified medicine, which is likely to involve a number of individual tests that could be collated into a prescribing algorithm for biological drug selection to be used in clinical practice.

OBJECTIVES

To quantify patient preferences for an algorithm-based approach to prescribing biologics ('biologic calculator') in psoriasis.

METHODS

An online survey comprising a discrete choice experiment (DCE) was conducted to elicit the preferences of two purposive samples of adults living with psoriasis in the UK, identified from a psoriasis patient organization (Psoriasis Association) and an online panel provider (Dynata). Respondents chose between two biologic calculators and conventional prescribing described using five attributes: treatment delay; positive predictive value; negative predictive value; risk of infection; and cost saving to the National Health Service. Each participant selected their preferred alternative from six hypothetical choice sets. Additional data, including sociodemographic characteristics, were collected. Choice data were analysed using conditional logit and fully correlated random parameters logit models.

RESULTS

Data from 212 respondents (67 from the Psoriasis Association and 145 from Dynata) were analysed. The signs of all estimated coefficients were consistent with a priori expectations. Respondents had a strong preference for a high predictive accuracy and avoiding serious infection, but there was evidence of systematic differences in preferences between the samples.

CONCLUSIONS

This study indicates that individuals with psoriasis would value a biologic calculator and suggested that such a biologic calculator should have sufficient accuracy to predict future response and risk of serious infection from the biologic.

Treatment preferences for preventive interventions for rheumatoid arthritis: protocol of a mixed methods case study for the Innovative Medicines Initiative PREFER project

INTRODUCTION

Amidst growing consensus that stakeholder decision-making during drug development should be informed by an understanding of patient preferences, the Innovative Medicines Initiative project 'Patient Preferences in Benefit-Risk Assessments during the Drug Life Cycle' (PREFER) is developing evidence-based recommendations about how and when patient preferences should be integrated into the drug life cycle. This protocol describes a PREFER clinical case study which compares two preference elicitation methodologies across several populations and provides information about benefit-risk trade-offs by those at risk of rheumatoid arthritis (RA) for preventive interventions.

METHODS AND ANALYSIS

This mixed methods study will be conducted in three countries (UK, Germany, Romania) to assess preferences of (1) first-degree relatives (FDRs) of patients with RA and (2) members of the public. Focus groups using nominal group techniques (UK) and ranking surveys (Germany and Romania) will identify and rank key treatment attributes. Focus group transcripts will be analysed thematically using the framework method and average rank orders calculated. These results will inform the treatment attributes to be assessed in a survey including a discrete choice experiment (DCE) and a probabilistic threshold technique (PTT). The survey will also include measures of sociodemographic variables, health literacy, numeracy, illness perceptions and beliefs about medicines. The survey will be administered to (1) 400 FDRs of patients with RA (UK); (2) 100 FDRs of patients with RA (Germany); and (3) 1000 members of the public in each of UK, Germany and Romania. Logit-based approaches will be used to analyse the DCE and imputation and interval regression for the PTT.

ETHICS AND DISSEMINATION

This study has been approved by the London-Hampstead Research Ethics Committee (19/LO/0407) and the Ethics Committee of the Friedrich-Alexander-Universität Erlangen-Nürnberg (92_17 B). The protocol has been approved by the PREFER expert review board. The results will be disseminated widely and will inform the PREFER recommendations.

A Picture is Worth a Thousand Words: The Role of Survey Training Materials in Stated-Preference Studies

Background

Online survey-based methods are increasingly used to elicit preferences for healthcare. This digitization creates an opportunity for interactive survey elements, potentially improving respondents’ understanding and/or engagement.

Objective

Our objective was to understand whether, and how, training materials in a survey influenced stated preferences.

Methods

An online discrete-choice experiment (DCE) was designed to elicit public preferences for a new targeted approach to prescribing biologics (“biologic calculator”) for rheumatoid arthritis (RA) compared with conventional prescribing. The DCE presented three alternatives, two biologic calculators and a conventional approach (opt out), described by five attributes: delay to treatment, positive predictive value, negative predictive value, infection risk, and cost saving to the national health service. Respondents were randomized to receive training materials as plain text or an animated storyline. Training materials contained information about RA and approaches to treatment and described the biologic calculator. Background questions included sociodemographics and self-reported measures of task difficulty and attribute non-attendance. DCE data were analyzed using conditional and heteroskedastic conditional logit (HCL) models.

Results

In total, 300 respondents completed the DCE, receiving either plain text (n = 158) or the animated storyline (n = 142). The HCL showed the estimated coefficients for all attributes aligned with a priori expectations and were statistically significant. The scale term was statistically significant, indicating that respondents who received plain-text materials had more random choices. Further tests suggested preference homogeneity after accounting for differences in scale.

Conclusions

Using animated training materials did not change the preferences of respondents, but they appeared to improve choice consistency, potentially allowing researchers to include more complex designs with increased numbers of attributes, levels, alternatives or choice sets.

Electronic supplementary material

The online version of this article (10.1007/s40271-019-00391-w) contains supplementary material, which is available to authorized users.

Benefit-Risk or Risk-Benefit Trade-Offs? Another Look at Attribute Ordering Effects in a Pilot Choice Experiment

BACKGROUND

Studies recommend randomising the order of attributes in discrete choice experiments (DCEs) to avoid bias; however, in a benefit-risk setting, this may increase the cognitive burden of respondents who compare the benefits and risks of treatments, or may affect their decision-making process. Based on these concerns, this paper explored attribute ordering effects in a benefit-risk DCE.

METHOD

Attribute ordering effects were explored in a large pilot DCE relating to the medical treatment of insomnia. Participants were randomised to one of three presentation orders: (1) benefits were presented before risks (BR); (2) risks were presented before benefits (RB); (3) all attributes were randomised (RN). For the RB and BR presentation orders, attributes were randomised within benefits and risks. Responses were assessed in three ways. First, variations in respondents' self-reported choice certainty were obtained. Second, variations in failure rates of stability and dominance tests were calculated. Third, a heteroscedastic error component model tested for differences in choice consistency across the three attribute orderings.

RESULTS

The final analysis included 156 respondents (RN: 54; BR: 49; RB: 53). No differences were found between the presentation orders with respect to stated choice certainty, or the proportion of respondents failing either the dominance or stability test. However, deterministic attribute grouping was associated with higher choice consistency.

CONCLUSION

To increase choice consistency, DCE attributes should be randomised within logical groups that may be further randomised to reduce the risk of ordering effects.

Use of Patient Preference Information in Benefit-Risk Assessment, Health Technology Assessment, and Pricing and Reimbursement Decisions: A Systematic Literature Review of Attempts and Initiatives

Objectives: Inclusion of patient preference (PP) data in decision making has been largely discussed in recent years. Healthcare decision makers—regulatory and health technology assessment (HTA)—are more and more conscious of the need for a patient-centered approach to decide on optimal allocation of scarce money, time, and technological resources. This literature review aims to examine the use of and recommendations for the integration of PP in decision making.

Methods: A literature search was conducted through PubMed/Medline in May 2019 to identify publications on PP studies used to inform benefit–risk assessments (BRAs) and HTAs and patient-centered projects and guidelines related to the inclusion of PPs in health policy decision making. After title and abstract screening and full-text review, selected publications were analyzed to retrieve data related to the collection, use, and/or submission of PPs informing BRA or HTA as well as attempts and initiatives in recommendations for PPs integration in decision-making processes.

Results: Forty-nine articles were included: 24 attempts and pilot project discussions and 25 PP elicitation studies. Quantitative approaches, particularly discrete choice experiments, were the most used (24 quantitative elicitation studies and 1 qualitative study). The objective of assessing PPs was to prioritize outcome-specific information, to value important treatment characteristics, to provide patient-focused benefit–risk trade-offs, and to appraise the patients' willingness to pay for new technologies. Moreover, attempts and pilot projects to integrate PPs in BRAs and HTAs were identified at the European level and across countries, but no clear recommendations have been issued yet. No less than seven public and/or private initiatives have been undertaken by governmental agencies and independent organizations to set guidance targeting improvement of patients' involvement in decision making.

Conclusion: Despite the initiatives undertaken, the pace of progress remains slow. The use of PPs remains poorly implemented, and evidence of proper use of these data in decision making is lacking. Guidelines and recommendations formalizing the purpose of collecting PPs, what methodology should be adopted and how, and who should be responsible for generating these data throughout the decision-making processes are needed to improve and empower integration of PPs in BRA and HTA.

Utilising Patient and Public Involvement in Stated Preference Research in Health: Learning from the Existing Literature and a Case Study

Publications reporting discrete choice experiments of healthcare interventions rarely discuss whether patient and public involvement (PPI) activities have been conducted. This paper presents examples from the existing literature and a detailed case study from the National Institute for Health Research-funded PATHWAY programme that comprehensively included PPI activities at multiple stages of preference research. Reflecting on these examples, as well as the wider PPI literature, we describe the different stages at which it is possible to effectively incorporate PPI across preference research, including the design, recruitment and dissemination of projects. Benefits of PPI activities include gaining practical insights from a wider perspective, which can positively impact experiment design as well as survey materials. Further benefits included advice around recruitment and reaching a greater audience with dissemination activities, amongst others. There are challenges associated with PPI activities; examples include time, cost and outlining expectations. Overall, although we acknowledge practical difficulties associated with PPI, this work highlights that it is possible for preference researchers to implement PPI across preference research. Further research systematically comparing methods related to PPI in preference research and their associated impact on the methods and results of studies would strengthen the literature.

Reporting Quality of Marginal Rates of Substitution in Discrete Choice Experiments That Elicit Patient Preferences

BACKGROUND

Discrete choice experiments (DCEs) are commonly used to elicit patient preferences as marginal rates of substitution (MRSs) between treatment or health service attributes. Because these studies are increasing in importance, it is vital that uncertainty around MRS estimates is reported.

OBJECTIVE

To review recently published DCE studies that elicit patient preferences in relation to MRS reporting and to explore the accuracy of using other reported information to estimate the uncertainty of the MRSs.

METHODS

A systematic literature review of DCEs conducted with patients between 2014 and July 2019 was performed. The number of studies reporting coefficients, MRSs, standard errors (SEs), and confidence intervals was recorded. If all information was reported, studies were included in an analysis to determine the impact of estimating the SEs of MRSs using coefficients and assuming zero covariance, to determine the impact of this assumption.

RESULTS

Two hundred and thirty-two patient DCEs were identified in the review; 34.1% (n = 79) reported 1 or more MRS and, of these, only 62.0% (n = 49) provided an estimate of the uncertainty. Of these studies, 16 contained enough information for inclusion in the analysis, providing 116 datapoints. Actual SEs were smaller than estimated SEs in 75.0% of cases (n = 87), and estimated SEs were within 25% of the actual SE in 59.5% of cases (n = 69).

CONCLUSION

Uncertainty of MRS estimates is unreported in a substantial proportion of recently published DCE studies. Estimating the SE of a MRS by solely using the SEs of the utility coefficients is likely to lead to biased estimates of the precision of patient trade-offs.

The importance of understanding patient and physician preferences for psoriasis treatment characteristics: a systematic review of discrete-choice experiments

Introduction: Treatment adherence continues to be a major challenge in psoriasis. Patient preference studies, especially discrete-choice experiments, are gaining popularity to gather insights into patient reported treatment outcomes. This systematic literature review aimed to critically assess all discrete choice experiments exploring patients' and physicians' preferences for psoriasis treatment characteristics.Methods: PubMed and EMBASE databases were searched using keywords "psoriasis" and "preferences" to identify relevant literature. Discrete-choice experiments conducted in French or English from the year 2000 onwards, that focused on evaluating psoriasis treatment preferences in patients and/or physicians, were included. The relative importance of treatment attributes was assessed and studies were critically appraised using validated checklists.Results: Out of 987 articles identified, 25 articles fulfilled the inclusion criteria. Overall, patients and physicians prioritize efficacy-specific outcomes. Patients are shown to place greater importance to process attributes when compared to physicians, especially route and location of administration. Physicians focus primarily of efficacy attributes, however when the top two attributes are considered, safety outcomes increasingly become considered important. Of the studies, 60% conducted subgroup analysis, of which many reported associations between specific patient characteristics and preferences. Factors such as age, disease severity, and duration of condition significantly affected preferences for treatment attributes.Conclusions: This review provides insight into the types of attributes that patients and physicians value most, and therefore can help improve shared decision-making. The findings of this study also encourage regulatory agencies to continue integrating patient preferences in their decision-making.

Integrating the Voice of the Patient Into the Medical Device Regulatory Process Using Patient Preference Information

The US Food and Drug Administration is one of several US and global agencies making strides to incorporate patient preference information (PPI) into its decision making. PPI has been included in 5 completed medical device marketing decisions to date. Its usage is not more widespread because of uncertainty about how to design "fit-for-purpose" patient preference studies and a lack of standards for the choice of preference elicitation methods, among other reasons. To advance the application of PPI to decision making about medical devices, the Food and Drug Administration has published a guidance document, "Patient Preference Information-Voluntary Submission, Review in Premarket Approval Applications, Humanitarian Device Exemption Applications, and De Novo Requests, and Inclusion in Decision Summaries and Device Labeling." This article discusses key concepts in the guidance document, in addition to providing lessons learned from the use of PPI for medical device regulatory applications to date and identifying new opportunities to leverage PPI to elevate the patient voice in the medical device product life cycle.

How Are Debriefing Questions Used in Health Discrete Choice Experiments? An Online Survey

OBJECTIVES

Debriefing questions can assess if respondents understand discrete choice experiments (DCEs) and are answering in a way consistent with theories of decision making and utility maximization. Nevertheless, there is limited literature about how often debriefing questions are included or how the results are used in health economics. The aim of this study was to conduct a survey of the frequency, type, and analysis of debriefing questions in health DCEs.

METHODS

We conducted an online survey of authors of published health DCEs, asking about their use of debriefing questions, including frequency, type, and analysis. We descriptively analyzed the sample characteristics and responses. Free-text questions were analyzed with qualitative thematic analysis.

RESULTS

We received 70 responses (43% response rate), of which 50% reported using debriefing questions. They were most commonly designed to assess difficulty (91%), understanding (49%), and attribute nonattendance (31%) rather than learning effects (3%) or monotonicity (11%). On average, 37% of debriefing questions were analyzed (range, 0% to 69%), and the results were used <50% of the time, usually to exclude respondents or interpret overall results. Researcher experience or confidence with DCEs did not affect their use of debriefing questions.

CONCLUSIONS

These results suggest that although half of researchers conducting health DCEs use debriefing questions, many do not analyze, use, or report the responses. Given the additional respondent burden, there is a need for reliable and valid debriefing questions. In the meantime, the inclusion, analysis, and reporting of debriefing questions should be carefully considered before DCE implementation.

Using discrete choice experiments to develop and deliver patient-centered psychological interventions: a systematic review

Developing and/or tailoring psychological interventions to align with patient preferences is a critical component of patient-centered care and has the potential to improve patient engagement and treatment outcomes. Discrete choice experiments (DCEs) are a quantitative method of assessing patient preferences that offer numerous strengths (i.e., ability to account for trade-offs), but are not routinely incorporated into health psychology coursework, likely leaving many unaware of the potential benefits of this methodology. To highlight the potential applications of DCEs within health psychology, this systematic review synthesises previous efforts to utilise DCEs to inform the design of patient-centered psychological care, defined as interventions targeting psychological (e.g., depression, anxiety) or behavioural health (e.g., pain management, adherence) concerns. Literature searches were conducted in March 2017 and November 2019 for articles reporting on DCEs using the terms 'discrete choice', 'conjoint', or 'stated preference'. Thirty-nine articles met all inclusion criteria and used DCEs to understand patient preferences regarding psychosocial clinical services (n = 12), lifestyle behaviour change interventions (n = 11), HIV prevention and/or intervention services (n = 10), disease self-management programmes (n = 4), or other interventions (n = 2). Clinical implications as well as limitations and directions for future research are discussed.

An Exploratory Application of Eye-Tracking Methods in a Discrete Choice Experiment

Background. Discrete choice experiments (DCEs) are increasingly used to elicit preferences for benefit-risk tradeoffs. The primary aim of this study was to explore how eye-tracking methods can be used to understand DCE respondents’ decision-making strategies. A secondary aim was to explore if the presentation and communication of risk affected respondents’ choices. Method. Two versions of a DCE were designed to understand the preferences of female members of the public for breast screening that varied in how risk attributes were presented. Risk was communicated as either 1) percentages or 2) icon arrays and percentages. Eye-tracking equipment recorded eye movements 1000 times a second. A debriefing survey collected sociodemographics and self-reported attribute nonattendance (ANA) data. A heteroskedastic conditional logit model analyzed DCE data. Eye-tracking data on pupil size, direction of motion, and total visual attention (dwell time) to predefined areas of interest were analyzed using ordinary least squares regressions. Results. Forty women completed the DCE with eye-tracking. There was no statistically significant difference in attention (fixations) to attributes between the risk communication formats. Respondents completing either version of the DCE with the alternatives presented in columns made more horizontal (left-right) saccades than vertical (up-down). Eye-tracking data confirmed self-reported ANA to the risk attributes with a 40% reduction in mean dwell time to the “probability of detecting a cancer” (P = 0.001) and a 25% reduction to the “risk of unnecessary follow-up” (P = 0.008). Conclusion. This study is one of the first to show how eye-tracking can be used to understand responses to a health care DCE and highlighted the potential impact of risk communication on respondents’ decision-making strategies. The results suggested self-reported ANA to cost attributes may not be reliable.

"I Was Trying to Do the Maths": Exploring the Impact of Risk Communication in Discrete Choice Experiments

BACKGROUND

Risk is increasingly used as an attribute in discrete choice experiments (DCEs). However, risk and probabilities are complex concepts that can be open to misinterpretation, potentially undermining the robustness of DCEs as a valuation method. This study aimed to understand how respondents made benefit-risk trade-offs in a DCE and if these were affected by the communication of the risk attributes.

METHODS

Female members of the public were recruited via local advertisements to participate in think-aloud interviews when completing a DCE eliciting their preferences for a hypothetical breast screening programme described by three attributes: probability of detecting a cancer; risk of unnecessary follow-up; and cost of screening. Women were randomised to receive risk information as either (1) percentages or (2) percentages and icon arrays. Interviews were digitally recorded then transcribed to generate qualitative data for thematic analysis.

RESULTS

Nineteen women completed the interviews (icon arrays n = 9; percentages n = 10). Analysis revealed four key themes where women made references to (1) the nature of the task; (2) their feelings; (3) their experiences, for instance making analogies to similar risks; and (4) economic phenomena such as opportunity costs and discounting.

CONCLUSION

Most women completed the DCE in line with economic theory; however, violations were identified. Women appeared to visualise risk whether they received icon arrays or percentages only. Providing clear instructions and graphics to aid interpretation of risk and qualitative piloting to verify understanding is recommended. Further investigation is required to determine if the process of verbalising thoughts changes the behaviour of respondents.

Preferences for aspects of antenatal and newborn screening: a systematic review

BACKGROUND

Many countries offer screening programmes to unborn and newborn babies (antenatal and newborn screening) to identify those at risk of certain conditions to aid earlier diagnosis and treatment. Technological advances have stimulated the development of screening programmes to include more conditions, subsequently changing the information required and potential benefit-risk trade-offs driving participation. Quantifying preferences for screening programmes can provide programme commissioners with data to understand potential demand, the drivers of this demand, information provision required to support the programmes and the extent to which preferences differ in a population. This study aimed to identify published studies eliciting preferences for antenatal and newborn screening programmes and provide an overview of key methods and findings.

METHODS

A systematic search of electronic databases for key terms identified eligible studies (discrete choice experiments (DCEs) or best-worst scaling (BWS) studies related to antenatal/newborn testing/screening published between 1990 and October 2018). Data were systematically extracted, tabulated and summarised in a narrative review.

RESULTS

A total of 19 studies using a DCE or BWS to elicit preferences for antenatal (n = 15; 79%) and newborn screening (n = 4; 21%) programmes were identified. Most of the studies were conducted in Europe (n = 12; 63%) but there were some examples from North America (n = 2; 11%) and Australia (n = 2; 11%). Attributes most commonly included were accuracy of screening (n = 15; 79%) and when screening occurred (n = 13; 68%). Other commonly occurring attributes included information content (n = 11; 58%) and risk of miscarriage (n = 10; 53%). Pregnant women (n = 11; 58%) and healthcare professionals (n = 11; 58%) were the most common study samples. Ten studies (53%) compared preferences across different respondents. Two studies (11%) made comparisons between countries. The most popular analytical model was a standard conditional logit model (n = 11; 58%) and one study investigated preference heterogeneity with latent class analysis.

CONCLUSION

There is an existing literature identifying stated preferences for antenatal and newborn screening but the incorporation of more sophisticated design and analytical methods to investigate preference heterogeneity could extend the relevance of the findings to inform commissioning of new screening programmes.

Discrete Choice Experiments in Health Economics: Past, Present and Future

OBJECTIVES

Discrete choice experiments (DCEs) are increasingly advocated as a way to quantify preferences for health. However, increasing support does not necessarily result in increasing quality. Although specific reviews have been conducted in certain contexts, there exists no recent description of the general state of the science of health-related DCEs. The aim of this paper was to update prior reviews (1990-2012), to identify all health-related DCEs and to provide a description of trends, current practice and future challenges.

METHODS

A systematic literature review was conducted to identify health-related empirical DCEs published between 2013 and 2017. The search strategy and data extraction replicated prior reviews to allow the reporting of trends, although additional extraction fields were incorporated.

RESULTS

Of the 7877 abstracts generated, 301 studies met the inclusion criteria and underwent data extraction. In general, the total number of DCEs per year continued to increase, with broader areas of application and increased geographic scope. Studies reported using more sophisticated designs (e.g. D-efficient) with associated software (e.g. Ngene). The trend towards using more sophisticated econometric models also continued. However, many studies presented sophisticated methods with insufficient detail. Qualitative research methods continued to be a popular approach for identifying attributes and levels.

CONCLUSIONS

The use of empirical DCEs in health economics continues to grow. However, inadequate reporting of methodological details inhibits quality assessment. This may reduce decision-makers' confidence in results and their ability to act on the findings. How and when to integrate health-related DCE outcomes into decision-making remains an important area for future research.

Patients' preferences in periodontal disease treatment elicited alongside an IQWiG benefit assessment: a feasibility study

BACKGROUND AND PURPOSE

The German Institute for Quality and Efficiency in Health Care (IQWiG) previously tested two preference elicitation methods in pilot projects and regarded them as generally feasible for prioritizing outcome-specific results of benefit assessment. The present study aimed to investigate the feasibility of completing a discrete choice experiment (DCE) within 3 months and to determine the relative importance of attributes of periodontal disease and its treatment.

PATIENTS AND METHODS

This preference elicitation was conducted alongside the IQWiG benefit assessment of systematic treatments of periodontal diseases. Attributes were defined based on the benefit assessment, literature review, and patients' and periodontologists' interviews. The DCE survey was completed by patients with a history of periodontal disease. Preferences were elicited for the attributes "tooth loss within next 10 years", "own costs for treatment, follow-up visits, re-treatment", "complaints and symptoms", and "frequency of follow-up visits". Patients completed a self-administered questionnaire including 12 choice tasks. Data were analyzed using a random parameters logit model. The relative attribute importance was calculated based on level ranges.

RESULTS

Within 3 months, survey development, data collection among 267 patients, data analysis, and provision of a study report could be completed. The analysis showed that tooth loss (score 0.73) was the most important attribute in patients' decisions, followed by complaints and symptoms (0.22), frequency of follow-up visits (0.02), and costs (0.03) (relative importance scores summing up to 1).

CONCLUSION

A preference analysis performing a DCE can be generally feasible within 3 months; however, a good research infrastructure and access to patients is required. Outcomes used in benefit assessments might need to be adapted to be used in preference analyses.

Feasibility of visual aids for risk evaluation by hospitalized patients with coronary artery disease: results from face-to-face interviews

PURPOSE

Communicating information about risk and probability to patients is considered a difficult task. In this study, we aim to evaluate the use of visual aids representing perioperative mortality and long-term survival in the communication process for patients diagnosed with coronary artery disease at the National Institute of Cardiology, a Brazilian public hospital specializing in cardiology.

PATIENTS AND METHODS

One-on-one interviews were conducted between August 1 and November 20, 2017. Patients were asked to imagine that their doctor was seeking their input in the decision regarding which treatment represented the best option for them. Patients were required to choose between alternatives by considering only the different benefits and risks shown in each scenario, described as the proportion of patients who had died during the perioperative period and within 5 years. Each participant evaluated the same eight scenarios. We evaluated their answers in a qualitative and quantitative analysis.

RESULTS

The main findings were that all patients verbally expressed concern about perioperative mortality and that 25% did not express concern about long-term mortality. Twelve percent considered the probabilities irrelevant on the grounds that their prognosis would depend on "God's will." Ten percent of the patients disregarded the reported likelihood of perioperative mortality, deciding to focus solely on the "chance of being cured." In the quantitative analysis, the vast majority of respondents chose the "correct" alternatives, meaning that they made consistent and rational choices.

CONCLUSION

The use of visual aids to present risk attributes appeared feasible in our sample. The impact of heuristics and religious beliefs on shared health decision making needs to be explored better in future studies.