Using the Threshold Technique to Elicit Patient Preferences: An Introduction to the Method and an Overview of Existing Empirical Applications

Multidimensional Thresholding for Individual-Level Preference Elicitation

OBJECTIVES

Multiple methods are available for collecting health preference information. However, information on the design and analysis of novel methods is limited. This article aims to provide the first introduction into the design and analysis of multidimensional thresholding (MDT).

METHODS

We introduce MDT as a 2-step approach: First, participants rank the largest possible improvements in all considered attributes by their importance. Second, participants complete a series of systematically combined trade-off questions. Hit-and-Run sampling is used for obtaining preference weights. We also use a computational experiment to compare different MDT designs.

RESULTS

The outlined MDT can generate preference information suitable for specifying a multiattribute utility function at the individual level. The computational experiment demonstrates the method's ability to recover preference weights at a high level of precision. While all designs in the computation experiment perform comparably well on average, the design outlined in the paper stands out with a high level of precision even if differences in relative attribute importance are large.

CONCLUSION

MDT is suitable for preference elicitation, in particular if sample sizes are small. Future research should help improve the methods (e.g., remove the need for an initial ranking) to increase the potential reach of MDT.

From Qualitative Research to Quantitative Preference Elicitation: An Example in Invasive Meningococcal Disease

BACKGROUND

Qualitative research is fundamental for designing discrete choice experiments (DCEs) but is often underreported in the preference literature. We developed a DCE to elicit preferences for vaccination against invasive meningococcal disease (IMD) among adolescents and young people (AYP) and parents and legal guardians (PLG) in the United States. This article reports the targeted literature review and qualitative interviews that informed the DCE design and demonstrates how to apply the recent reporting guidelines for qualitative developmental work in preference studies.

METHODS

This study included two parts: a targeted literature review and qualitative interviews. The Medline and Embase databases were searched for quantitative and qualitative studies on IMD and immunization. The results of the targeted literature review informed a qualitative interview guide. Sixty-minute, online, semi-structured interviews with AYP and PLG were used to identify themes related to willingness to be vaccinated against IMD. Participants were recruited through a third-party recruiter's database and commercial online panels. Interviews included vignettes about IMD and vaccinations and three thresholding exercises examining the effect of incidence rate, disability rate, and fatality rate on vaccination preferences. Participant responses related to the themes were counted.

RESULTS

The targeted literature review identified 31 concepts that were synthesized into six topics for the qualitative interviews. Twenty AYP aged 16-23 years and 20 PLG of adolescents aged 11-17 years were interviewed. Four themes related to willingness to be vaccinated emerged: attitudes towards vaccination, knowledge and information, perception of IMD, and vaccine attributes. Most participants were concerned about IMD (AYP 60%; PLG 85%) and had positive views of vaccination (AYP 80%; PLG 60%). Ninety percent of AYP and 75% of PLG always chose vaccination over no vaccination, independent of IMD incidence rate, disability rate, or fatality rate.

CONCLUSION

Willingness to be vaccinated against IMD was affected by vaccine attributes but largely insensitive to IMD incidence and severity. This article provides an example of how to apply the recent reporting guidelines for qualitative developmental work in preference studies, with 21 out of 22 items in the guidelines being considered.

Making Use of Technology to Improve Stated Preference Studies

The interest in quantifying stated preferences for health and healthcare continues to grow, as does the technology available to support and improve health preference studies. Technological advancements in the last two decades have implications and opportunities for preference researchers designing, administering, analysing, interpreting and applying the results of stated preference surveys. In this paper, we summarise selected technologies and how these can benefit a preference study. We discuss empirical evaluations of the technology in preference research, with examples from health where possible. The technologies reviewed include serious games, virtual reality, eye tracking, innovative formats and decision aids with values clarification components. We conclude with a critical reflection on the benefits and limitations of implementing (often costly) technology alongside stated preference studies.

Overweight and obese women's symptoms, knowledge, and preferences regarding endometrial biopsy for endometrial cancer detection: A threshold technique survey

Background

The incidence of endometrial cancer (EC) in the United States continues to rise, driven mainly by the obesity epidemic. We sought to determine overweight and obese women's cancer risk knowledge and preferences regarding diagnostic endometrial biopsy (EMB) for EC detection.

Methods

An online survey was administered to overweight and obese women without EC recruited through the electronic medical record's online patient portal. Baseline questions queried gynecologic history, cancer risk knowledge, and factors potentially influencing decision-making for EMB. We used the threshold survey technique to identify the minimum acceptable risk (MAR) threshold at which each respondent would be willing to undergo an EMB to detect EC.

Results

Of 357 respondents (median age 45 years (interquartile range [IQR]: 38-54); median BMI 39 [IQR: 36.0-44.6]), fewer than half (48.7 %) were aware that obesity is a risk factor for EC, and 10 % considered their risk of EC to be high. Almost half (42 %) of respondents reported MAR thresholds characterized as very low (0-1 %), and these were more common among respondents with higher BMIs. Forty percent identified their weight as a factor influencing their MAR threshold decision, while 76 % identified their perceived personal risk as a factor. Less than half cited immediate risks of the procedure.

Conclusion

Many patients reported being willing to undergo an EMB at very low risk thresholds for EC. Perceived personal risk is a stronger factor in decision-making than immediate procedural risks. Providers should focus on communicating patients' risk to motivate EMB to detect EC where appropriate.

Preferences of Patients with Amyotrophic Lateral Sclerosis for Intrathecal Drug Delivery: Choosing between an Implanted Drug-Delivery Device and Therapeutic Lumbar Puncture

BACKGROUND

Novel intrathecal treatments for amyotrophic lateral sclerosis (ALS) may require delivery using lumbar puncture (LP). Implanted drug-delivery devices (IDDDs) could be an alternative but little is known about patients' preferences for intrathecal drug-delivery methods.

OBJECTIVE

We aimed to elicit preferences of patients with ALS for routine LP and IDDD use.

METHODS

A discrete choice experiment (DCE) and a threshold technique (TT) exercise were conducted online among patients with ALS in the US and Europe. In the DCE, patients made trade-offs between administration attributes. Attributes were identified from qualitative interviews. The TT elicited maximum acceptable risks (MARs) of complications from device implantation surgery. DCE data were analyzed using mixed logit to quantify relative attribute importance (RAI) as the maximum contribution of each attribute to a preference, and to estimate MARs of device failure. TT data were analyzed using interval regression. Four scenarios of LP and IDDD were compared.

RESULTS

Participants (N = 295) had a mean age of 57.7 years; most (74.2%) were diagnosed < 3 years ago. Preferences were affected by device failure risk (RAI 28.6%), administration frequency (26.4%), administration risk (19.7%), overall duration (17.8%), and appointment location (7.5%). Patients accepted a 5.6% device failure risk to reduce overall duration from 2 h to 30 min and a 3.6% risk for administration in a local clinic instead of a hospital. The average MAR of complications from implantation surgery was 29%. Patients preferred IDDD over LP in three of four scenarios.

CONCLUSION

Patients considered an IDDD as a valuable alternative to LP in multiple clinical settings.

Participatory Value Evaluation (PVE): A New Preference-Elicitation Method for Decision Making in Healthcare

Participatory value evaluation (PVE) has recently been introduced in the field of health as a new method to elicit stated preferences for public policies. PVE is a method in which respondents in a choice experiment are presented with various policy options and their attributes, and are asked to compose their portfolio of preference given a public-resource constraint. This paper aims to illustrate PVE's potential for informing healthcare decision making and to position it relative to established preference-elicitation methods. We first describe PVE and its theoretical background. Next, by means of a narrative review of the eight existing PVE applications within and outside the health domain, we illustrate the different implementations of the main features of the method. We then compare PVE to several established preference-elicitation methods in terms of the structure and nature of the choice tasks presented to respondents. The portfolio-based choice task in a PVE requires respondents to consider a set of policy alternatives in relation to each other and to make trade-offs subject to one or more constraints, which more closely resembles decision making by policymakers. When using a flexible budget constraint, respondents can trade-off their private income with public expenditures. Relative to other methods, a PVE may be cognitively more demanding and is less efficient; however, it seems a promising complementary method for the preference-based assessment of health policies. Further research into the feasibility and validity of the method is required before researchers and policymakers can fully appreciate the advantages and disadvantages of the PVE as a preference-elicitation method.

Attitudes of healthcare professionals and drug regulators about progression-free survival as endpoint in the advanced cancer setting

PURPOSE

To describe the attitudes of healthcare professionals and drug regulators about progression-free survival (PFS) as efficacy endpoint in clinical trials with patients with advanced cancer and to explore to what extent these attitudes influence the willingness to trade between PFS and toxicity.

METHODS

Cross-sectional survey with regulators from the European Medicines Agency (EMA), and healthcare professionals (HCP) from the "Stichting Hemato-Oncologie voor Volwassenen Nederland" (HOVON) collaborative group and the European Organisation for Research and Treatment of Cancer (EORTC). Attitudes towards PFS were elicited using 5-point Likert items. The respondents' willingness to trade between PFS and grade 3 or 4 (G34) toxicity was assessed using the threshold technique and quantified in terms of their maximum acceptable risk (MAR).

RESULTS

Responses were collected from 287 HCPs and 64 regulators with mainly clinical expertise. Attitudes towards PFS were often spread out in both groups and related to beliefs about PFS being a likely surrogate for clinical benefit, being an intrinsic benefit to be distinguished from OS, or on the importance given to OS. Being a regulator or holding stronger beliefs about PFS being a likely surrogate or an intrinsic benefit were associated with a higher MAR. Presence of a supportive trend in OS was stated as important but was not associated with MAR. There was agreement on the need to address bias in the adjudication of PFS and the need for improving communication to patients about meaning, strengths, and limitations of improvements in PFS.

CONCLUSION

Attitudes towards PFS were spread out and were associated with individual differences in the willingness to trade between toxicity and PFS. There was agreement on the need to address bias in the adjudication of PFS and improving communication to patients.

Two Methods, One Story? Comparing Results of a Choice Experiment and Multidimensional Thresholding From a Clinician Preference Study in Aneurysmal Subarachnoid Hemorrhage

OBJECTIVES

An increasing number of methods are used to elicit health preference information. It is unclear whether different elicitation methods produce similar results and policy advice. Here, we compared the results from a discrete choice experiment (DCE) and multidimensional thresholding (MDT) that were conducted in the same sample.

METHODS

Clinicians (N = 350) completed a DCE and MDT to elicit their preferences for 4 attributes related to the medical management of subarachnoid hemorrhage after aneurysm repair. Preference weights were compared between the DCE and MDT using a complete combinatorial convolution test. Additionally, data from the DCE and MDT were used to compute preference-based net treatment values for 16 hypothetical treatment profiles versus 1000 simulated comparators. The implied treatment recommendations were compared between the DCE and MDT.

RESULTS

Preference weight distributions and median weights did not differ significantly between the DCE and MDT for any attribute: likelihood of delayed cerebral ischemia (medians 0.48 vs 0.40; P = .41), risk of lung complications (medians 0.27 vs 0.30; P = .52), risk of hypotension (medians 0.10 vs 0.11; P = .55), and risk of anemia (medians 0.07 vs 0.07; P = .50). The DCE and MDT produced similar treatment net value distributions (P > .05) and implied the same treatment recommendations in 82.3% of cases.

CONCLUSIONS

The DCE and MDT elicited similar preference distributions and produced the same treatment recommendations for most tested cases. However, the share of people supporting the average treatment recommendation differed. More research is needed to determine how these findings would compare with those in other populations (in particular, patients) and applications.

Maximum Acceptable Risk Estimation Based on a Discrete Choice Experiment and a Probabilistic Threshold Technique

OBJECTIVE

We aimed to empirically compare maximum acceptable risk results estimated using both a discrete choice experiment (DCE) and a probabilistic threshold technique (PTT).

METHODS

Members of the UK general public (n = 982) completed an online survey including a DCE and a PTT (in random order) measuring their preferences for preventative treatment for rheumatoid arthritis. For the DCE, a Bayesian D-efficient design consisting of four blocks of 15 choice tasks was constructed including six attributes with varying levels. The PTT used identical risk and benefit attributes. For the DCE, a panel mixed-logit model was conducted, both mean and individual estimates were used to calculate maximum acceptable risk. For the PTT, interval regression was used to calculate maximum acceptable risk. Perceived complexity of the choice tasks and preference heterogeneity were investigated for both methods.

RESULTS

Maximum acceptable risk confidence intervals of both methods overlapped for serious infection and serious side effects but not for mild side effects (maximum acceptable risk was 32.7 percent-points lower in the PTT). Although, both DCE and PTT tasks overall were considered easy or very easy to understand and answer, significantly more respondents rated the DCE choice tasks as easier to understand compared with those who rated the PTT as easier (7-percentage point difference; p < 0.05).

CONCLUSIONS

Maximum acceptable risk estimate confidence intervals based on a DCE and a PTT overlapped for two out of the three included risk attributes. More respondents rated the DCE as easier to understand. This may suggest that the DCE is better suited in studies estimating maximum acceptable risk for multiple risk attributes of differing severity, while the PTT may be better suited when measuring heterogeneity in maximum acceptable risk estimates or when investigating one or more serious adverse events.

Planning Benefit-Risk Assessments Using Visualizations

A Benefit Risk Assessment Plan (BRAP) describes the assessments planned to determine whether the benefits of an investigational drug outweigh the risks. The plan can have two sections, one with timelines for aligning resources with decision milestones and the other for pre-specifying assessments for decision milestones. Regulatory guidance recommends a proactive planning process over an ad-hoc process. However, very little has been published about proactive plans themselves. This article works through a hypothetical example visualizing a series of assessments across the drug development lifecycle. Based on a regulatory framework, the planning process starts with assessing the medical condition and current treatment options. These early assessments bring out major considerations in assessing the investigational drug.

Eliciting preferences for cancer screening tests: Comparison of a discrete choice experiment and the threshold technique

OBJECTIVE

To compare results of three preference elicitation methods for a cancer screening test.

METHODS

Participants (undergraduate students) completed a discrete choice experiment (DCE) and a threshold technique (TT) task. Accuracy (false positives, false negatives), benefits (lives saved), and cost for a cancer screening test were used as attributes in the DCE and branching logic for the TT. Participants were also asked a direct elicitation question regarding a hypothetical screening test for breast (women) or prostate (men) cancer without mortality benefit. Correlations assessed the relationship between DCE and TT thresholds. Thresholds were standardized and ranked for both methods to compare. A logistic regression used the thresholds to predict results of the direct elicitation.

RESULTS

DCE and TT estimates were not meaningfully correlated (max ρ = 0.17). Participant rankings of attributes matched only 20% of the time (58/292). Neither method predicted preference for being screened (ps > 0.21).

CONCLUSIONS

The DCE and TT yielded different preference estimates (and rank orderings) for the same participant. Neither method predicted patients' desires for a screening test.

PRACTICE IMPLICATIONS

Clinicians, patients, policy makers, and researchers should be aware that patient preference results may be sensitive to the method of eliciting preferences.

Increasing Access to Buprenorphine for Opioid Use Disorder in Primary Care: an Assessment of Provider Incentives

BACKGROUND

Primary care providers (PCPs) are essential to increasing access to office-based buprenorphine medication treatment for opioid use disorder (B-MOUD). Barriers to B-MOUD prescribing are well-documented, but there is little information regarding incentives to overcome these barriers.

OBJECTIVE

To identify optimal incentives for PCPs to promote B-MOUD prescribing and compare incentive preferences across provider and practice characteristics.

DESIGN

We surveyed PCPs using best-worst scaling (BWS) to prioritize seven potential incentives for B-MOUD prescribing (monetary compensation, paid vacation, protected time, professional development, reduced workload, service recognition, clinical resources). We then used a direct elicitation approach to determine preferred incentive levels (e.g., monetary thresholds) and types (e.g., specific clinical resources).

PARTICIPANTS

Primary care physicians and advanced practice providers (APPs) at a large Department of Veterans Affairs healthcare system.

MAIN MEASURES

B-MOUD prescribing incentive preferences and relative preference levels using descriptive statistics and conditional logistic regression with relative importance scale transformation (coefficients sum to 100, higher coefficient=greater importance).

KEY RESULTS

Fifty-three PCPs responded (73% response), including 47% APPs and 36% from community-based clinics. Reduced workload (relative importance score=26.8), protected time (18.7), and clinical resources (16.8) were significantly more preferred (Ps < 0.001) than professional development (10.5), paid vacation (10.3), or service recognition (1.5). Relative importance of monetary compensation varied between physicians (12.6) and APPs (17.5) and between PCPs located at a medical center (11.4) versus community clinic (22.3). APPs were more responsive than physicians to compensation increases of $5000 and $12,000 but less responsive to $25,000; trends were similar for medical center versus community clinic PCPs. The most frequently requested clinical resource was on-demand consult access to an addiction specialist.

CONCLUSIONS

Interventions promoting workload reductions, protected time, and clinical resources could increase access to B-MOUD in primary care. Monetary incentives may be additionally needed to improve B-MOUD prescribing among APPs and within community clinics.

Quantitative Benefit-Risk Assessment in Medical Product Decision Making: A Good Practices Report of an ISPOR Task Force

Benefit-risk assessment is commonly conducted by drug and medical device developers and regulators, to evaluate and communicate issues around benefit-risk balance of medical products. Quantitative benefit-risk assessment (qBRA) is a set of techniques that incorporate explicit outcome weighting within a formal analysis to evaluate the benefit-risk balance. This report describes emerging good practices for the 5 main steps of developing qBRAs based on the multicriteria decision analysis process. First, research question formulation needs to identify the needs of decision makers and requirements for preference data and specify the role of external experts. Second, the formal analysis model should be developed by selecting benefit and safety endpoints while eliminating double counting and considering attribute value dependence. Third, preference elicitation method needs to be chosen, attributes framed appropriately within the elicitation instrument, and quality of the data should be evaluated. Fourth, analysis may need to normalize the preference weights, base-case and sensitivity analyses should be conducted, and the effect of preference heterogeneity analyzed. Finally, results should be communicated efficiently to decision makers and other stakeholders. In addition to detailed recommendations, we provide a checklist for reporting qBRAs developed through a Delphi process conducted with 34 experts.

Suitability of Preference Methods Across the Medical Product Lifecycle: A Multicriteria Decision Analysis

OBJECTIVES

This study aimed to understand the importance of criteria describing methods (eg, duration, costs, validity, and outcomes) according to decision makers for each decision point in the medical product lifecycle (MPLC) and to determine the suitability of a discrete choice experiment, swing weighting, probabilistic threshold technique, and best-worst scale cases 1 and 2 at each decision point in the MPLC.

METHODS

Applying multicriteria decision analysis, an online survey was sent to MPLC decision makers (ie, industry, regulatory, and health technology assessment representatives). They ranked and weighted 19 methods criteria from an existing performance matrix about their respective decisions across the MPLC. All criteria were given a relative weight based on the ranking and rating in the survey after which an overall suitability score was calculated for each preference elicitation method per decision point. Sensitivity analyses were conducted to reflect uncertainty in the performance matrix.

RESULTS

Fifty-nine industry, 29 regulatory, and 5 health technology assessment representatives completed the surveys. Overall, "estimating trade-offs between treatment characteristics" and "estimating weights for treatment characteristics" were highly important criteria throughout all MPLC decision points, whereas other criteria were most important only for specific MPLC stages. Swing weighting and probabilistic threshold technique received significantly higher suitability scores across decision points than other methods. Sensitivity analyses showed substantial impact of uncertainty in the performance matrix.

CONCLUSION

Although discrete choice experiment is the most applied preference elicitation method, other methods should also be considered to address the needs of decision makers. Development of evidence-based guidance documents for designing, conducting, and analyzing such methods could enhance their use.

A Roadmap for Increasing the Usefulness and Impact of Patient-Preference Studies in Decision Making in Health: A Good Practices Report of an ISPOR Task Force

Many qualitative and quantitative methods are readily available to study patient preferences in health. These methods are now being used to inform a wide variety of decisions, and there is a growing body of evidence showing studies of patient preferences can be used for decision making in a wide variety of contexts. This ISPOR Task Force report synthesizes current good practices for increasing the usefulness and impact of patient-preference studies in decision making. We provide the ISPOR Roadmap for Patient Preferences in Decision Making that invites patient-preference researchers to work with decision makers, patients and patient groups, and other stakeholders to ensure that studies are useful and impactful. The ISPOR Roadmap consists of 5 key elements: (1) context, (2) purpose, (3) population, (4) method, and (5) impact. In this report, we define these 5 elements and provide good practices on how patient-preference researchers and others can actively contribute to increasing the usefulness and impact of patient-preference studies in decision making. We also present a set of key questions that can support researchers and other stakeholders (eg, funders, reviewers, readers) to assess efforts that promote the ongoing impact (both intended and unintended) of a particular preference study and additional studies in the future.

Risk thresholds for patients to switch between daily tablets and biweekly infusions in second-line treatment for advanced hepatocellular carcinoma: a patient preference study

BACKGROUND

Historically, high hepatocellular carcinoma (HCC)-related mortality has been, in part, due to lack of effective therapies; however, several systemic therapies have been recently approved for HCC treatment, including regorafenib and ramucirumab. These two treatments utilize different routes of administration (four daily tablets and biweekly intravenous infusions, respectively) and have different risks of adverse events (AEs). However, we lack data on patient preferences in balancing the route of administration and risk of AEs in patients with HCC. We aimed to determine patient preferences and trade-offs for second-line treatment in patients with HCC.  METHODS: Patients with advanced or metastatic HCC were recruited through their physicians for this study. Patient preferences were assessed by using a modified threshold technique (TT) design in which respondents were asked two direct-elicitation questions before (assuming same safety and efficacy and only varying mode of administration) and after (incorporating the safety profiles of ramucirumab and regorafenib) the TT series on seven risks of clinically relevant AEs.

RESULTS

In total, of the 157 patients recruited by their physicians, 150 were eligible and consented to participate. In the first elicitation question (assuming risk and efficacy were equivalent), 61.3% of patients preferred daily tablets. However, 76.7% of patients preferred the biweekly infusion when the safety profiles of the two available second-line therapies were included. The TT analysis confirmed that preferences for oral administration were not strong enough to balance out the risk of AEs that differentiate the two therapies.

DISCUSSION

We found that when patients were asked to choose between a daily, oral medication and a biweekly IV medication for HCC, they were more likely to choose a daily, oral medication if efficacy and safety profiles were the same. However, when risks of AEs representing the safety profiles of two currently available second-line treatments were introduced in a second direct-elicitation question, respondents often selected an IV administration with a safety profile similar to ramucirumab, rather than oral tablets with a safety profile similar to regorafenib. Our findings indicate that the risk profile of a second-line treatment for HCC may be more important than the mode of administration to patients.

A world of choices: preference elicitation methods for improving the delivery and uptake of HIV prevention and treatment

PURPOSE OF REVIEW

Despite the growing availability of effective HIV prevention and treatment interventions, there are large gaps in their uptake and sustained use across settings. It is crucial to elicit and apply patients' and stakeholders' preferences to maximize the impact of existing and future interventions. This review summarizes quantitative preference elicitation methods (PEM) and how they can be applied to improve the delivery and uptake of HIV prevention and treatment interventions.

RECENT FINDINGS

PEM are increasingly applied in HIV implementation research; however, discrete choice experiments (DCEs) have predominated. Beyond DCEs, there are other underutilized PEM that may improve the reach and effectiveness of HIV prevention and treatment interventions among individuals by prioritizing their barriers to engagement and determining which attributes of interventions and delivery strategies are most valued. PEM can also enhance the adoption and sustained implementation of strategies to deliver HIV prevention and treatment interventions by assessing which attributes are the most acceptable and appropriate to key stakeholders.

SUMMARY

Greater attention to and incorporation of patient's and stakeholders' preferences for HIV prevention and treatment interventions and their delivery has the potential to increase the number of persons accessing and retained in HIV prevention and treatment services.

A simulated maximum likelihood procedure for analyzing imprecise trade-off thresholds between the benefits and harms of medicines

Stated preference studies in which information on the willingness to trade-off between the benefits and harms of medicines is elicited from patients or other stakeholders are becoming increasingly mainstream. Such trade-offs can mathematically be represented by a weighted additive function, with the weights, whose ratios determine how much an individual is willing to trade-off between the treatment attributes, being the response vector for the statistical analysis. One way of eliciting trade-off information is through multi-dimensional thresholding (MDT), which is a bisection-based approach that results in increasingly tight bounds on the values of the weights ratios. While MDT is cognitively less demanding than other, more direct elicitation methods, its use complicates the statistical analysis as it results in weights data that are region censored. In this article, we present a simulated maximum likelihood (SML) procedure for fitting a Dirichlet population model directly to the region-censored weights data and perform a series of computational experiments to compare the proposed SML procedure to a naive approach in which a Dirichlet distribution is fitted to the centroids of the weights boundaries obtained with MDT. The results indicate that the SML procedure consistently outperformed the centroid-based approach, with the centroid-based approach requiring three bisection steps per trade-off to achieve a similar precision as the SML procedure with one bisection step per trade-off. Using the newly proposed SML procedure, MDT can be applied with smaller sample sizes or with fewer questions compared to the more naïve centroid-based approach that was applied in previous applications of MDT.

Acceptable risks of treatments to prevent rheumatoid arthritis among first-degree relatives: demographic and psychological predictors of risk tolerance

OBJECTIVES

To quantify tolerance to risks of preventive treatments among first-degree relatives (FDRs) of patients with rheumatoid arthritis (RA).

METHODS

Preventive treatments for RA are under investigation. In a preference survey, adult FDRs assumed a 60% chance of developing RA within 2 years and made choices between no treatment and hypothetical preventive treatment options with a fixed level of benefit (reduction in chance of developing RA from 60% to 20%) and varying levels of risks. Using a probabilistic threshold technique, each risk was increased or decreased until participants switched their choice. Perceived risk of RA, health literacy, numeracy, Brief Illness Perception Questionnaire and Beliefs about Medicines Questionnaire-General were also assessed. Maximum acceptable risk (MAR) was summarised using descriptive statistics. Associations between MARs and participants' characteristics were assessed using interval regression with effects coding.

RESULTS

289 FDRs (80 male) responded. The mean MAR for a 40% reduction in chance of developing RA was 29.08% risk of mild side effects, 9.09% risk of serious infection and 0.85% risk of a serious side effect. Participants aged over 60 years were less tolerant of serious infection risk (mean MAR ±2.06%) than younger participants. Risk of mild side effects was less acceptable to participants who perceived higher likelihood of developing RA (mean MAR ±3.34%) and more acceptable to those believing that if they developed RA it would last for a long time (mean MAR ±4.44%).

CONCLUSIONS

Age, perceived chance of developing RA and perceived duration of RA were associated with tolerance to some risks of preventive RA therapy.

Travelers' preferences for tick-borne encephalitis vaccination

BACKGROUND

This study aimed to quantify preferences and risk tolerance for a tick-borne encephalitis (TBE) vaccination.

RESEARCH DESIGN AND METHODS

A stated-preference survey instrument was administered to international travelers living in the United States to elicit preferences for a no-cost TBE vaccine when planning an international trip, conditional upon four different qualitative levels of endemic TBE risk.

RESULTS

The likelihood of choosing the vaccine increased with a destination's level of endemic risk. Most respondents (94%) would choose to receive the vaccine at the highest risk level presented in the survey (i.e. when multiple TBE cases among humans are reported year after year); 6% of the sample would choose not to receive the vaccine at any risk level. Respondents who engage in outdoor activities were twice as likely as the average respondent to choose vaccination rather than opting out of vaccination, and were one-third more likely than the average respondent to choose to receive the vaccine at the lowest risk level.

CONCLUSIONS

Respondents were highly interested in a TBE vaccine, assuming no cost, and most were willing to be vaccinated at all qualitative TBE risk levels. Respondents who participated in outdoor activities were more likely than the average respondent to choose the vaccine.

Preference Testing in Medical Devices: Current Framework and Regulatory Gaps

Preference testing is a valuable source of information that can be provided by both healthcare professionals (HCPs) and patients (users). It can be used to improve the design and development of medical devices by feeding into device usability and, ultimately, risk management. Furthermore, it can aid with selecting the most appropriate clinical endpoints to be used in the clinical evaluation of a device and increase patient engagement by incorporating patient-relevant outcomes. Preference testing is widely conducted in the food industry but is not widespread in the medical field due to limited guidelines and a lack of regulatory framework. As such, manufacturers may be unaware of the benefits of preference testing and fail to take full advantage of it, or conversely, may use inappropriate methodology and/or analyses and consequently fail to collect meaningful data. In this position paper, we aim to highlight the benefits and uses of preference testing, along with potential methods that could be used for preference testing of medical devices. A key step towards the wider implementation of preference testing in medical devices is for the publication of international standards and guidelines for the collection, assessment, and implementation of preference data into the life cycle of a medical device.

Development of a Patient Preference Survey for Wearable Kidney Replacement Therapy Devices

Background

Recent innovations have the potential to disrupt the current paradigm for kidney failure treatment. The US Food and Drug Administration is committed to incorporating valid scientific evidence about how patients weigh the benefits and risks of new devices into their decision making, but to date, premarket submission of patient preference information (PPI) has been limited for kidney devices. With input from stakeholders, we developed a survey intended to yield valid PPI, capturing how patients trade off the potential benefits and risks of wearable dialysis devices and in-center hemodialysis.

Methods

We conducted concept elicitation interviews with individuals receiving dialysis to inform instrument content. After instrument drafting, we conducted two rounds of pretest interviews to evaluate survey face validity, comprehensibility, and perceived relevance. We pilot tested the survey with in-center hemodialysis patients to assess comprehensibility and usability further. Throughout, we used participant input to guide survey refinements.

Results

Thirty-six individuals receiving in-center or home dialysis participated in concept elicitation (N=20) and pretest (N=16) interviews. Participants identified reduced fatigue, lower treatment burden, and enhanced freedom as important benefits of a wearable device, and many expressed concerns about risks related to device disconnection-specifically bleeding and infection. We drafted a survey that included descriptions of the risks of serious bleeding and serious infection and an assessment of respondent willingness to wait for a safer device. Input from pretest interviewees led to various instrument modifications, including treatment descriptions, item wording, and risk-level explanations. Pilot testing of the updated survey among 24 in-center hemodialysis patients demonstrated acceptable survey comprehensibility and usability, although 50% of patients required some assistance.

Conclusions

The final survey is a 54-item web-based instrument that will yield estimates of the maximal acceptable risk for the described wearable device and willingness to wait for wearable devices with lower risk.

Accounting for Preference Heterogeneity in Discrete-Choice Experiments: An ISPOR Special Interest Group Report

OBJECTIVES

Discrete choice experiments (DCEs) are increasingly used to elicit preferences for health and healthcare. Although many applications assume preferences are homogenous, there is a growing portfolio of methods to understand both explained (because of observed factors) and unexplained (latent) heterogeneity. Nevertheless, the selection of analytical methods can be challenging and little guidance is available. This study aimed to determine the state of practice in accounting for preference heterogeneity in the analysis of health-related DCEs, including the views and experiences of health preference researchers and an overview of the tools that are commonly used to elicit preferences.

METHODS

An online survey was developed and distributed among health preference researchers and nonhealth method experts, and a systematic review of the DCE literature in health was undertaken to explore the analytical methods used and summarize trends.

RESULTS

Most respondents (n = 59 of 70, 84%) agreed that accounting for preference heterogeneity provides a richer understanding of the data. Nevertheless, there was disagreement on how to account for heterogeneity; most (n = 60, 85%) stated that more guidance was needed. Notably, the majority (n = 41, 58%) raised concern about the increasing complexity of analytical methods. Of the 342 studies included in the review, half (n = 175, 51%) used a mixed logit with continuous distributions for the parameters, and a third (n = 110, 32%) used a latent class model.

CONCLUSIONS

Although there is agreement about the importance of accounting for preference heterogeneity, there are noticeable disagreements and concerns about best practices, resulting in a clear need for further analytical guidance.

Patient preference for early onset of efficacy of preventive migraine treatments

Objective

The objective of this study was to ascertain to what extent adults with migraine value an early onset of efficacy for preventive migraine treatments.

Background

In placebo‐controlled clinical trials, treatment with eptinezumab resulted in a lower proportion of adults with migraine on the first day following infusion (day 1; 14% point‐reduction for chronic migraine [CM] in PROMISE‐2 and 8% point‐reduction for episodic migraine [EM] in PROMISE‐1).

Methods

Adults with migraine completed an online preference‐elicitation thresholding exercise to ascertain to what extent they value not having a migraine on day 1 postdosing relative to a clinically relevant reduction in number of migraine days during the first month postdosing (≥2 migraine‐free days for CM and ≥1 migraine‐free days for EM).

Results

One hundred and one participants (mean age, 50.6 ± 12.4 years; 81 [80%] women) were included. In participants with CM, 29 of 50 (58%) considered the eptinezumab‐generated reduction in the likelihood of migraine on day 1 postdosing to be at least as important as a clinically relevant reduction in number of migraine days the first month postdosing, whereas 37 of 50 (74%) considered a clinically relevant reduction of migraine days the first month postdosing to have a value equivalent to the eptinezumab‐generated reduction in the likelihood of migraine on day 1 postdosing. In participants with EM, 18 of 35 (51%) considered the eptinezumab‐generated reduction in the likelihood of migraine on day 1 postdosing to be at least as important as a clinically relevant reduction in migraine days the first month postdosing, whereas 24 of 35 (69%) considered a clinically relevant reduction of migraine days the first month postdosing to have a value equivalent to the eptinezumab‐generated reduction in the likelihood of migraine on day 1 postdosing.

Conclusion

Most participants considered the reduction in the likelihood of migraine offered by eptinezumab on day 1 postdosing to be at least as important as a clinically relevant reduction in migraine days the first month postdosing.

A Systematic Review of Discrete Choice Experiments in Oncology Treatments

BACKGROUND

As the number and type of cancer treatments available rises and patients live with the consequences of their disease and treatments for longer, understanding preferences for cancer care can help inform decisions about optimal treatment development, access, and care provision. Discrete choice experiments (DCEs) are commonly used as a tool to elicit stakeholder preferences; however, their implementation in oncology may be challenging if burdensome trade-offs (e.g. length of life versus quality of life) are involved and/or target populations are small.

OBJECTIVES

The aim of this review was to characterise DCEs relating to cancer treatments that were conducted between 1990 and March 2020.

DATA SOURCES

EMBASE, MEDLINE, and the Cochrane Database of Systematic Reviews were searched for relevant studies.

STUDY ELIGIBILITY CRITERIA

Studies were included if they implemented a DCE and reported outcomes of interest (i.e. quantitative outputs on participants' preferences for cancer treatments), but were excluded if they were not focused on pharmacological, radiological or surgical treatments (e.g. cancer screening or counselling services), were non-English, or were a secondary analysis of an included study.

ANALYSIS METHODS

Analysis followed a narrative synthesis, and quantitative data were summarised using descriptive statistics, including rankings of attribute importance.

RESULT

Seventy-nine studies were included in the review. The number of published DCEs relating to oncology grew over the review period. Studies were conducted in a range of indications (n = 19), most commonly breast (n =10, 13%) and prostate (n = 9, 11%) cancer, and most studies elicited preferences of patients (n = 59, 75%). Across reviewed studies, survival attributes were commonly ranked as most important, with overall survival (OS) and progression-free survival (PFS) ranked most important in 58% and 28% of models, respectively. Preferences varied between stakeholder groups, with patients and clinicians placing greater importance on survival outcomes, and general population samples valuing health-related quality of life (HRQoL). Despite the emphasis of guidelines on the importance of using qualitative research to inform attribute selection and DCE designs, reporting on instrument development was mixed.

LIMITATIONS

No formal assessment of bias was conducted, with the scope of the paper instead providing a descriptive characterisation. The review only included DCEs relating to cancer treatments, and no insight is provided into other health technologies such as cancer screening. Only DCEs were included.

CONCLUSIONS AND IMPLICATIONS

Although there was variation in attribute importance between responder types, survival attributes were consistently ranked as important by both patients and clinicians. Observed challenges included the risk of attribute dominance for survival outcomes, limited sample sizes in some indications, and a lack of reporting about instrument development processes.

PROTOCOL REGISTRATION

PROSPERO 2020 CRD42020184232.

Patient preferences for gene therapy in haemophilia: Results from the PAVING threshold technique survey

OBJECTIVES

The aim of the Patient preferences to Assess Value IN Gene therapies (PAVING) study was to investigate trade-offs that adult Belgian people with haemophilia (PWH) A and B are willing to make when choosing between prophylactic factor replacement therapy (PFRT) and gene therapy.

METHODS

The threshold technique was used to quantify the minimum acceptable benefit (MAB) of a switch from PFRT to gene therapy in terms of 'Annual bleeding rate' (ABR), 'Chance to stop prophylaxis' (STOP), and 'Quality of life' (QOL). The design was supported by stakeholder involvement and included an educational tool on gene therapy. Threshold intervals were analysed using interval regression models in Stata 16.

RESULTS

A total of 117 PWH completed the survey. Mean thresholds were identified for all benefits, but substantial preference heterogeneity was observed; especially for the STOP thresholds, where the distribution of preferences was bimodal. Time spent on the educational tool and residence were found to impact MAB thresholds. The most accepted (88% of PWH) gene therapy profile investigated in this study comprised of zero bleeds per year (vs. six for PFRT), 90% chance to stop prophylaxis, no impact on QoL, and 10 years of follow-up on side effects (vs. 30 for PFRT).

CONCLUSIONS

Results from this study proved the value of educating patients on novel treatments. Moreover, preference heterogeneity for novel treatments was confirmed in this study. In gene therapy decision-making, preference heterogeneity and the impact of patient education on acceptance should be considered.

Treatment preferences for preventive interventions for rheumatoid arthritis: protocol of a mixed methods case study for the Innovative Medicines Initiative PREFER project

INTRODUCTION

Amidst growing consensus that stakeholder decision-making during drug development should be informed by an understanding of patient preferences, the Innovative Medicines Initiative project 'Patient Preferences in Benefit-Risk Assessments during the Drug Life Cycle' (PREFER) is developing evidence-based recommendations about how and when patient preferences should be integrated into the drug life cycle. This protocol describes a PREFER clinical case study which compares two preference elicitation methodologies across several populations and provides information about benefit-risk trade-offs by those at risk of rheumatoid arthritis (RA) for preventive interventions.

METHODS AND ANALYSIS

This mixed methods study will be conducted in three countries (UK, Germany, Romania) to assess preferences of (1) first-degree relatives (FDRs) of patients with RA and (2) members of the public. Focus groups using nominal group techniques (UK) and ranking surveys (Germany and Romania) will identify and rank key treatment attributes. Focus group transcripts will be analysed thematically using the framework method and average rank orders calculated. These results will inform the treatment attributes to be assessed in a survey including a discrete choice experiment (DCE) and a probabilistic threshold technique (PTT). The survey will also include measures of sociodemographic variables, health literacy, numeracy, illness perceptions and beliefs about medicines. The survey will be administered to (1) 400 FDRs of patients with RA (UK); (2) 100 FDRs of patients with RA (Germany); and (3) 1000 members of the public in each of UK, Germany and Romania. Logit-based approaches will be used to analyse the DCE and imputation and interval regression for the PTT.

ETHICS AND DISSEMINATION

This study has been approved by the London-Hampstead Research Ethics Committee (19/LO/0407) and the Ethics Committee of the Friedrich-Alexander-Universität Erlangen-Nürnberg (92_17 B). The protocol has been approved by the PREFER expert review board. The results will be disseminated widely and will inform the PREFER recommendations.

Patients' and caregivers' maximum acceptable risk of death for non-curative gene therapy to treat Duchenne muscular dystrophy

BACKGROUND

Gene therapy offers an etiologically targeted treatment for genetic disorders. Little is known about the acceptance of mortality risk among patients with progressive, fatal conditions. We assessed patients' and caregivers' maximum acceptable risk (MAR) of mortality for gene therapy when used to treat Duchenne muscular dystrophy (DMD).

METHODS

The threshold technique was used to assess tolerance for mortality risks using a hypothetical vignette. Gene therapy was described as non-curative and resulting in a slowing of progression and with a 10-year benefit duration. MAR was analyzed using interval regression for gene therapy initiated "now"; in the last year of walking well; in the last year of being able to bring arms to mouth; and in newborns (for caregivers only).

RESULTS

Two hundred eighty-five caregivers and 35 patients reported the greatest MAR for gene therapy initiated in last year of being able to lift arms (mean MAR 6.3%), followed by last year of walking well (mean MAR 4.4%), when used "now" (mean MAR 3.5%), and when used in the newborn period (mean MAR 2.1%, caregivers only). About 35% would accept ≥200/2000 risk in the last year of being able to lift arms. Non-ambulatory status predicted accepting 1.8 additional points in MAR "now" compared with ambulatory status (p = 0.010). Respondent type (caregiver or patient) did not predict MAR.

CONCLUSION

In this first quantitative study to assess MAR associated with first-generation DMD gene therapy, we find relatively high tolerance for mortality risk in response to a non-curative treatment scenario. Risk tolerance increased with disease progression. Patients and caregivers did not have significantly different MAR. These results have implications for protocol development and shared decision making.

Risk tolerance in adjuvant and metastatic melanoma settings: a patient perspective study using the threshold technique

Background: Adverse events (e.g., pyrexia) may affect treatment patterns and adherence. This study explored pyrexia risk tolerance among melanoma patients when treatment benefit is unknown versus known. Materials & methods: US respondents with stage III (n = 100) or stage III unresectable/stage IV melanoma (n = 125) chose between hypothetical melanoma treatments, defined by reoccurrence/progression-free survival and pyrexia risk, one resembling standard-of-care and one resembling dabrafenib + trametinib. Respondents chose first when efficacy was unknown and then when efficacy was known; pyrexia risk was varied systematically to define maximum acceptable risk. Results: Maximum acceptable risk of pyrexia was statistically significantly higher when efficacy was known versus unknown in stage III patients (85 vs 34%) and stage III unresectable/stage IV patients (66 vs 57%). Conclusion: Patients accepted higher levels of pyrexia risk when they understood treatment benefit.

Patient Preferences to Assess Value IN Gene Therapies: Protocol Development for the PAVING Study in Hemophilia

Introduction: Gene therapies are innovative therapies that are increasingly being developed. However, health technology assessment (HTA) and payer decision making on these therapies is impeded by uncertainties, especially regarding long-term outcomes. Through measuring patient preferences regarding gene therapies, the importance of unique elements that go beyond health gain can be quantified and inform value assessments. We designed a study, namely the Patient preferences to Assess Value IN Gene therapies (PAVING) study, that can inform HTA and payers by investigating trade-offs that adult Belgian hemophilia A and B patients are willing to make when asked to choose between a standard of care and gene therapy.

Methods and Analysis: An eight-step approach was taken to establish the protocol for this study: (1) stated preference method selection, (2) initial attributes identification, (3) stakeholder (HTA and payer) needs identification, (4) patient relevant attributes and information needs identification, (5) level identification and choice task construction, (6) educational tool design, (7) survey integration, and (8) piloting and pretesting. In the end, a threshold technique survey was designed using the attributes “Annual bleeding rate,” “Chance to stop prophylaxis,” “Time that side effects have been studied,” and “Quality of Life.”

Ethics and Dissemination: The Medical Ethics Committee of UZ KU Leuven/Research approved the study. Results from the study will be presented to stakeholders and patients at conferences and in peer-reviewed journals. We hope that results from the PAVING study can inform decision makers on the acceptability of uncertainties and the value of gene therapies to patients.

Parkinson's Patients' Tolerance for Risk and Willingness to Wait for Potential Benefits of Novel Neurostimulation Devices: A Patient-Centered Threshold Technique Study

Background. Parkinson’s disease (PD) is neurodegenerative, causing motor, cognitive, psychological, somatic, and autonomic symptoms. Understanding PD patients’ preferences for novel neurostimulation devices may help ensure that devices are delivered in a timely manner with the appropriate level of evidence. Our objective was to elicit preferences and willingness-to-wait for novel neurostimulation devices among PD patients to inform a model of optimal trial design. Methods. We developed and administered a survey to PD patients to quantify the maximum levels of risks that patients would accept to achieve potential benefits of a neurostimulation device. Threshold technique was used to quantify patients’ risk thresholds for new or worsening depression or anxiety, brain bleed, or death in exchange for improvements in “on-time,” motor symptoms, pain, cognition, and pill burden. The survey elicited patients’ willingness to wait to receive treatment benefit. Patients were recruited through Fox Insight, an online PD observational study. Results. A total of 2740 patients were included and a majority were White (94.6%) and had a 4-year college degree (69.8%). Risk thresholds increased as benefits increased. Threshold for depression or anxiety was substantially higher than threshold for brain bleed or death. Patient age, ambulation, and prior neurostimulation experience influenced risk tolerance. Patients were willing to wait an average of 4 to 13 years for devices that provide different levels of benefit. Conclusions. PD patients are willing to accept substantial risks to improve symptoms. Preferences are heterogeneous and depend on treatment benefit and patient characteristics. The results of this study may be useful in informing review of device applications and other regulatory decisions and will be input into a model of optimal trial design for neurostimulation devices.

Reporting Quality of Marginal Rates of Substitution in Discrete Choice Experiments That Elicit Patient Preferences

BACKGROUND

Discrete choice experiments (DCEs) are commonly used to elicit patient preferences as marginal rates of substitution (MRSs) between treatment or health service attributes. Because these studies are increasing in importance, it is vital that uncertainty around MRS estimates is reported.

OBJECTIVE

To review recently published DCE studies that elicit patient preferences in relation to MRS reporting and to explore the accuracy of using other reported information to estimate the uncertainty of the MRSs.

METHODS

A systematic literature review of DCEs conducted with patients between 2014 and July 2019 was performed. The number of studies reporting coefficients, MRSs, standard errors (SEs), and confidence intervals was recorded. If all information was reported, studies were included in an analysis to determine the impact of estimating the SEs of MRSs using coefficients and assuming zero covariance, to determine the impact of this assumption.

RESULTS

Two hundred and thirty-two patient DCEs were identified in the review; 34.1% (n = 79) reported 1 or more MRS and, of these, only 62.0% (n = 49) provided an estimate of the uncertainty. Of these studies, 16 contained enough information for inclusion in the analysis, providing 116 datapoints. Actual SEs were smaller than estimated SEs in 75.0% of cases (n = 87), and estimated SEs were within 25% of the actual SE in 59.5% of cases (n = 69).

CONCLUSION

Uncertainty of MRS estimates is unreported in a substantial proportion of recently published DCE studies. Estimating the SE of a MRS by solely using the SEs of the utility coefficients is likely to lead to biased estimates of the precision of patient trade-offs.