MCDA swing weighting and discrete choice experiments for elicitation of patient benefit-risk preferences: a critical assessment

Eumycetoma causative agents: A systematic review to inform the World Health Organization priority list of fungal pathogens

The World Health Organization, in response to the growing burden of fungal disease, established a process to develop a fungal priority pathogens list. This systematic review aimed to evaluate the epidemiology and impact of eumycetoma. PubMed and Web of Science were searched to identify studies published between 1 January 2011 and 19 February 2021. Studies reporting on mortality, inpatient care, complications and sequelae, antifungal susceptibility, risk factors, preventability, annual incidence, global distribution, and emergence during the study time frames were selected. Overall, 14 studies were eligible for inclusion. Morbidity was frequent with moderate to severe impairment of quality of life in 60.3%, amputation in up to 38.5%, and recurrent or long-term disease in 31.8%-73.5% of patients. Potential risk factors included male gender (56.6%-79.6%), younger age (11-30 years; 64%), and farming occupation (62.1%-69.7%). Mycetoma was predominantly reported in Sudan, particularly in central Sudan (37%-76.6% of cases). An annual incidence of 0.1/100 000 persons and 0.32/100  000 persons/decade was reported in the Philippines and Uganda, respectively. In Uganda, a decline in incidence from 3.37 to 0.32/100  000 persons between two consecutive 10-year periods (2000-2009 and 2010-2019) was detected. A community-based, multi-pronged prevention programme was associated with a reduction in amputation rates from 62.8% to 11.9%. With the pre-specified criteria, no studies of antifungal drug susceptibility, mortality, and hospital lengths of stay were identified. Future research should include larger cohort studies, greater drug susceptibility testing, and global surveillance to develop evidence-based treatment guidelines and to determine more accurately the incidence and trends over time.

Aspergillus fumigatus-a systematic review to inform the World Health Organization priority list of fungal pathogens

Recognizing the growing global burden of fungal infections, the World Health Organization established a process to develop a priority list of fungal pathogens (FPPL). In this systematic review, we aimed to evaluate the epidemiology and impact of invasive infections caused by Aspergillus fumigatus to inform the first FPPL. The pre-specified criteria of mortality, inpatient care, complications and sequelae, antifungal susceptibility, risk factors, preventability, annual incidence, global distribution, and emergence were used to search for relevant articles between 1 January 2016 and 10 June 2021. Overall, 49 studies were eligible for inclusion. Azole antifungal susceptibility varied according to geographical regions. Voriconazole susceptibility rates of 22.2% were reported from the Netherlands, whereas in Brazil, Korea, India, China, and the UK, voriconazole susceptibility rates were 76%, 94.7%, 96.9%, 98.6%, and 99.7%, respectively. Cross-resistance was common with 85%, 92.8%, and 100% of voriconazole-resistant A. fumigatus isolates also resistant to itraconazole, posaconazole, and isavuconazole, respectively. The incidence of invasive aspergillosis (IA) in patients with acute leukemia was estimated at 5.84/100 patients. Six-week mortality rates in IA cases ranged from 31% to 36%. Azole resistance and hematological malignancy were poor prognostic factors. Twelve-week mortality rates were significantly higher in voriconazole-resistant than in voriconazole-susceptible IA cases (12/22 [54.5%] vs. 27/88 [30.7%]; P = .035), and hematology patients with IA had significantly higher mortality rates compared with solid-malignancy cases who had IA (65/217 [30%] vs. 14/78 [18%]; P = .04). Carefully designed surveillance studies linking laboratory and clinical data are required to better inform future FPPL.

Multidimensional Thresholding for Individual-Level Preference Elicitation

OBJECTIVES

Multiple methods are available for collecting health preference information. However, information on the design and analysis of novel methods is limited. This article aims to provide the first introduction into the design and analysis of multidimensional thresholding (MDT).

METHODS

We introduce MDT as a 2-step approach: First, participants rank the largest possible improvements in all considered attributes by their importance. Second, participants complete a series of systematically combined trade-off questions. Hit-and-Run sampling is used for obtaining preference weights. We also use a computational experiment to compare different MDT designs.

RESULTS

The outlined MDT can generate preference information suitable for specifying a multiattribute utility function at the individual level. The computational experiment demonstrates the method's ability to recover preference weights at a high level of precision. While all designs in the computation experiment perform comparably well on average, the design outlined in the paper stands out with a high level of precision even if differences in relative attribute importance are large.

CONCLUSION

MDT is suitable for preference elicitation, in particular if sample sizes are small. Future research should help improve the methods (e.g., remove the need for an initial ranking) to increase the potential reach of MDT.

Participatory Value Evaluation (PVE): A New Preference-Elicitation Method for Decision Making in Healthcare

Participatory value evaluation (PVE) has recently been introduced in the field of health as a new method to elicit stated preferences for public policies. PVE is a method in which respondents in a choice experiment are presented with various policy options and their attributes, and are asked to compose their portfolio of preference given a public-resource constraint. This paper aims to illustrate PVE's potential for informing healthcare decision making and to position it relative to established preference-elicitation methods. We first describe PVE and its theoretical background. Next, by means of a narrative review of the eight existing PVE applications within and outside the health domain, we illustrate the different implementations of the main features of the method. We then compare PVE to several established preference-elicitation methods in terms of the structure and nature of the choice tasks presented to respondents. The portfolio-based choice task in a PVE requires respondents to consider a set of policy alternatives in relation to each other and to make trade-offs subject to one or more constraints, which more closely resembles decision making by policymakers. When using a flexible budget constraint, respondents can trade-off their private income with public expenditures. Relative to other methods, a PVE may be cognitively more demanding and is less efficient; however, it seems a promising complementary method for the preference-based assessment of health policies. Further research into the feasibility and validity of the method is required before researchers and policymakers can fully appreciate the advantages and disadvantages of the PVE as a preference-elicitation method.

Comparing Discrete Choice Experiment with Swing Weighting to Estimate Attribute Relative Importance: A Case Study in Lung Cancer Patient Preferences

INTRODUCTION

Discrete choice experiments (DCE) are commonly used to elicit patient preferences and to determine the relative importance of attributes but can be complex and costly to administer. Simpler methods that measure relative importance exist, such as swing weighting with direct rating (SW-DR), but there is little empirical evidence comparing the two. This study aimed to directly compare attribute relative importance rankings and weights elicited using a DCE and SW-DR.

METHODS

A total of 307 patients with non-small-cell lung cancer in Italy and Belgium completed an online survey assessing preferences for cancer treatment using DCE and SW-DR. The relative importance of the attributes was determined using a random parameter logit model for the DCE and rank order centroid method (ROC) for SW-DR. Differences in relative importance ranking and weights between the methods were assessed using Cohen's weighted kappa and Dirichlet regression. Feedback on ease of understanding and answering the 2 tasks was also collected.

RESULTS

Most respondents (>65%) found both tasks (very) easy to understand and answer. The same attribute, survival, was ranked most important irrespective of the methods applied. The overall ranking of the attributes on an aggregate level differed significantly between DCE and SW-ROC (P < 0.01). Greater differences in attribute weights between attributes were reported in DCE compared with SW-DR (P < 0.01). Agreement between the individual-level attribute ranking across methods was moderate (weighted Kappa 0.53-0.55).

CONCLUSION

Significant differences in attribute importance between DCE and SW-DR were found. Respondents reported both methods being relatively easy to understand and answer. Further studies confirming these findings are warranted. Such studies will help to provide accurate guidance for methods selection when studying relative attribute importance across a wide array of preference-relevant decisions.

HIGHLIGHTS

Both DCEs and SW tasks can be used to determine attribute relative importance rankings and weights; however, little evidence exists empirically comparing these methods in terms of outcomes or respondent usability.Most respondents found the DCE and SW tasks very easy or easy to understand and answer.A direct comparison of DCE and SW found significant differences in attribute importance rankings and weights as well as a greater spread in the DCE-derived attribute relative importance weights.

The therapeutic threshold in clinical decision-making for TB

Because TB control is still hampered by the limitations of diagnostic tools, diagnostic uncertainty is common. The decision to offer treatment is based on clinical decision-making. The therapeutic threshold, test threshold and test-treatment threshold can guide in making these decisions. This review summarizes the literature on methods to estimate the therapeutic threshold that have been applied for TB. Only five studies estimated the threshold for the diagnosis of TB. The therapeutic threshold can be estimated by prescriptive methods, based on calculations, and by descriptive methods, deriving the threshold from observing clinical practice. Test and test-treatment thresholds can be calculated using the therapeutic threshold and the characteristics of an available diagnostic test. Estimates of the therapeutic threshold for pulmonary TB from intuitive descriptive approaches (20%-50%) are higher than theoretical prescriptive calculations (2%-3%). In conclusion, estimates of the therapeutic threshold for pulmonary TB depend on the method used. Other methods exist within the field of decision-making that have yet to be implemented or adapted as tools to estimate the TB therapeutic threshold. Because clinical decision-making is a core element of TB management, it is necessary to find a new, clinician-friendly way to unbiasedly estimate context-specific, agreed upon therapeutic thresholds.

Discrete choice experiment versus swing-weighting: A head-to-head comparison of diabetic patient preferences for glucose-monitoring devices

INTRODUCTION

Limited evidence exists for how patient preference elicitation methods compare directly. This study compares a discrete choice experiment (DCE) and swing-weighting (SW) by eliciting preferences for glucose-monitoring devices in a population of diabetes patients.

METHODS

A sample of Dutch adults with type 1 or 2 diabetes (n = 459) completed an online survey assessing their preferences for glucose-monitoring devices, consisting of both a DCE and a SW exercise. Half the sample completed the DCE first; the other half completed the SW first. For the DCE, the relative importance of the attributes of the devices was determined using a mixed-logit model. For the SW, the relative importance of the attributes was based on ranks and points allocated to the 'swing' from the worst to the best level of the attribute. The preference outcomes and self-reported response burden were directly compared between the two methods.

RESULTS

Participants reported they perceived the DCE to be easier to understand and answer compared to the SW. Both methods revealed that cost and precision of the device were the most important attributes. However, the DCE had a 14.9-fold difference between the most and least important attribute, while the SW had a 1.4-fold difference. The weights derived from the SW were almost evenly distributed between all attributes.

CONCLUSIONS

The DCE was better received by participants, and generated larger weight differences between each attribute level, making it the more informative method in our case study. This method comparison provides further evidence of the degree of method suitability and trustworthiness.

Taking the Shortcut: Simplifying Heuristics in Discrete Choice Experiments

Health-related discrete choice experiments (DCEs) information can be used to inform decision-making on the development, authorisation, reimbursement and marketing of drugs and devices as well as treatments in clinical practice. Discrete choice experiment is a stated preference method based on random utility theory (RUT), which imposes strong assumptions on respondent choice behaviour. However, respondents may use choice processes that do not adhere to the normative rationality assumptions implied by RUT, applying simplifying decision rules that are more selective in the amount and type of processed information (i.e., simplifying heuristics). An overview of commonly detected simplifying heuristics in health-related DCEs is lacking, making it unclear how to identify and deal with these heuristics; more specifically, how researchers might alter DCE design and modelling strategies to accommodate for the effects of heuristics. Therefore, the aim of this paper is three-fold: (1) provide an overview of common simplifying heuristics in health-related DCEs, (2) describe how choice task design and context as well as target population selection might impact the use of heuristics, (3) outline DCE design strategies that recognise the use of simplifying heuristics and develop modelling strategies to demonstrate the detection and impact of simplifying heuristics in DCE study outcomes.

Quantitative Benefit-Risk Assessment in Medical Product Decision Making: A Good Practices Report of an ISPOR Task Force

Benefit-risk assessment is commonly conducted by drug and medical device developers and regulators, to evaluate and communicate issues around benefit-risk balance of medical products. Quantitative benefit-risk assessment (qBRA) is a set of techniques that incorporate explicit outcome weighting within a formal analysis to evaluate the benefit-risk balance. This report describes emerging good practices for the 5 main steps of developing qBRAs based on the multicriteria decision analysis process. First, research question formulation needs to identify the needs of decision makers and requirements for preference data and specify the role of external experts. Second, the formal analysis model should be developed by selecting benefit and safety endpoints while eliminating double counting and considering attribute value dependence. Third, preference elicitation method needs to be chosen, attributes framed appropriately within the elicitation instrument, and quality of the data should be evaluated. Fourth, analysis may need to normalize the preference weights, base-case and sensitivity analyses should be conducted, and the effect of preference heterogeneity analyzed. Finally, results should be communicated efficiently to decision makers and other stakeholders. In addition to detailed recommendations, we provide a checklist for reporting qBRAs developed through a Delphi process conducted with 34 experts.

Suitability of Preference Methods Across the Medical Product Lifecycle: A Multicriteria Decision Analysis

OBJECTIVES

This study aimed to understand the importance of criteria describing methods (eg, duration, costs, validity, and outcomes) according to decision makers for each decision point in the medical product lifecycle (MPLC) and to determine the suitability of a discrete choice experiment, swing weighting, probabilistic threshold technique, and best-worst scale cases 1 and 2 at each decision point in the MPLC.

METHODS

Applying multicriteria decision analysis, an online survey was sent to MPLC decision makers (ie, industry, regulatory, and health technology assessment representatives). They ranked and weighted 19 methods criteria from an existing performance matrix about their respective decisions across the MPLC. All criteria were given a relative weight based on the ranking and rating in the survey after which an overall suitability score was calculated for each preference elicitation method per decision point. Sensitivity analyses were conducted to reflect uncertainty in the performance matrix.

RESULTS

Fifty-nine industry, 29 regulatory, and 5 health technology assessment representatives completed the surveys. Overall, "estimating trade-offs between treatment characteristics" and "estimating weights for treatment characteristics" were highly important criteria throughout all MPLC decision points, whereas other criteria were most important only for specific MPLC stages. Swing weighting and probabilistic threshold technique received significantly higher suitability scores across decision points than other methods. Sensitivity analyses showed substantial impact of uncertainty in the performance matrix.

CONCLUSION

Although discrete choice experiment is the most applied preference elicitation method, other methods should also be considered to address the needs of decision makers. Development of evidence-based guidance documents for designing, conducting, and analyzing such methods could enhance their use.

A world of choices: preference elicitation methods for improving the delivery and uptake of HIV prevention and treatment

PURPOSE OF REVIEW

Despite the growing availability of effective HIV prevention and treatment interventions, there are large gaps in their uptake and sustained use across settings. It is crucial to elicit and apply patients' and stakeholders' preferences to maximize the impact of existing and future interventions. This review summarizes quantitative preference elicitation methods (PEM) and how they can be applied to improve the delivery and uptake of HIV prevention and treatment interventions.

RECENT FINDINGS

PEM are increasingly applied in HIV implementation research; however, discrete choice experiments (DCEs) have predominated. Beyond DCEs, there are other underutilized PEM that may improve the reach and effectiveness of HIV prevention and treatment interventions among individuals by prioritizing their barriers to engagement and determining which attributes of interventions and delivery strategies are most valued. PEM can also enhance the adoption and sustained implementation of strategies to deliver HIV prevention and treatment interventions by assessing which attributes are the most acceptable and appropriate to key stakeholders.

SUMMARY

Greater attention to and incorporation of patient's and stakeholders' preferences for HIV prevention and treatment interventions and their delivery has the potential to increase the number of persons accessing and retained in HIV prevention and treatment services.

A simulated maximum likelihood procedure for analyzing imprecise trade-off thresholds between the benefits and harms of medicines

Stated preference studies in which information on the willingness to trade-off between the benefits and harms of medicines is elicited from patients or other stakeholders are becoming increasingly mainstream. Such trade-offs can mathematically be represented by a weighted additive function, with the weights, whose ratios determine how much an individual is willing to trade-off between the treatment attributes, being the response vector for the statistical analysis. One way of eliciting trade-off information is through multi-dimensional thresholding (MDT), which is a bisection-based approach that results in increasingly tight bounds on the values of the weights ratios. While MDT is cognitively less demanding than other, more direct elicitation methods, its use complicates the statistical analysis as it results in weights data that are region censored. In this article, we present a simulated maximum likelihood (SML) procedure for fitting a Dirichlet population model directly to the region-censored weights data and perform a series of computational experiments to compare the proposed SML procedure to a naive approach in which a Dirichlet distribution is fitted to the centroids of the weights boundaries obtained with MDT. The results indicate that the SML procedure consistently outperformed the centroid-based approach, with the centroid-based approach requiring three bisection steps per trade-off to achieve a similar precision as the SML procedure with one bisection step per trade-off. Using the newly proposed SML procedure, MDT can be applied with smaller sample sizes or with fewer questions compared to the more naïve centroid-based approach that was applied in previous applications of MDT.

What matters most to patients with multiple myeloma? A Pan-European patient preference study

INTRODUCTION

Given the rapid increase in novel treatments for patients with multiple myeloma (MM), this patient preference study aimed to establish which treatment attributes matter most to MM patients and evaluate discrete choice experiment (DCE) and swing weighting (SW) as two elicitation methods for quantifying patients' preferences.

METHODS

A survey incorporating DCE and SW was disseminated among European MM patients. The survey included attributes and levels informed by a previous qualitative study with 24 MM patients. Latent class and mixed logit models were used to estimate the DCE attribute weights and descriptive analyses were performed to derive SW weights. MM patients and patient organisations provided extensive feedback during survey development.

RESULTS

393 MM patients across 21 countries completed the survey (M years since diagnosis=6; M previous therapies=3). Significant differences (p<.01) between participants' attribute weights were revealed depending on participants' prior therapy experience, and their experience with side-effects and symptoms. Multivariate analyses showed that participants across the three MM patient classes identified via the latent class model differed regarding their past number of therapies (F=4.772, p=.009). Patients with the most treatments (class 1) and those with the least treatments (class 3) attached more value to life expectancy versus quality of life-related attributes such as pain, mobility and thinking problems. Conversely, patients with intermediary treatment experience (class 2) attached more value to quality of life-related attributes versus life expectancy. Participants highlighted the difficulty of trading-off between life expectancy and quality of life and between physical and mental health. Participants expressed a need for greater psychological support to cope with their symptoms, treatment side-effects, and uncertainties. With respect to patients' preferences for the DCE or SW questions, 42% had no preference, 32% preferred DCE, and 25% preferred SW.

CONCLUSIONS

Quality of life-related attributes affecting MM patients' physical, mental and psychological health such as pain, mobility and thinking problems were considered very important to MM patients, next to life expectancy. This underscores a need to include such attributes in decision-making by healthcare stakeholders involved in MM drug development, evidence generation, evaluation, and clinical practice. This study highlights DCE as the preferred methodology for understanding relative attribute weights from a patient's perspective.

Preference Testing in Medical Devices: Current Framework and Regulatory Gaps

Preference testing is a valuable source of information that can be provided by both healthcare professionals (HCPs) and patients (users). It can be used to improve the design and development of medical devices by feeding into device usability and, ultimately, risk management. Furthermore, it can aid with selecting the most appropriate clinical endpoints to be used in the clinical evaluation of a device and increase patient engagement by incorporating patient-relevant outcomes. Preference testing is widely conducted in the food industry but is not widespread in the medical field due to limited guidelines and a lack of regulatory framework. As such, manufacturers may be unaware of the benefits of preference testing and fail to take full advantage of it, or conversely, may use inappropriate methodology and/or analyses and consequently fail to collect meaningful data. In this position paper, we aim to highlight the benefits and uses of preference testing, along with potential methods that could be used for preference testing of medical devices. A key step towards the wider implementation of preference testing in medical devices is for the publication of international standards and guidelines for the collection, assessment, and implementation of preference data into the life cycle of a medical device.

Multi-stage Internet public opinion risk grading analysis of public health emergencies: An empirical study on Microblog in COVID-19

In the period of Corona Virus Disease 2019 (COVID-19), millions of people participate in the discussion of COVID-19 on the Internet, which can easily trigger public opinion and threaten social stability. This paper creatively proposes a multi-stage risk grading model of Internet public opinion for public health emergencies. On the basis of general public opinion risk grading analysis, the model continuously pays attention to the risk level of Internet public opinion based on the time scale of regular or major information updates. This model combines Analytic Hierarchy Process Sort II (AHPSort II) and Swing Weighting (SW) methods and proposes a new Multi-Criteria Decision Making (MCDM) method - AHPSort II-SW. Intuitionistic fuzzy number and linguistic fuzzy number are introduced into the model to evaluate the criteria that cannot be quantified. The multi-stage model is tested using more than 2,000 textual data about COVID-19 collected from Microblog, a leading social media platform in China. Seven public opinion risk assessments were conducted from January 23 to April 8, 2020. The empirical results show that in the early COVID-19 outbreak, the risk of public opinion is more serious on macroscopic view. In details, the risk of public opinion decreases slowly with time, but the emergence of important events may still increase the risk of public opinion. The analysis results are in line with the actual situation and verify the effectiveness of the method. Comparative analysis indicates the improved method is proved to be superior and effective, sensitivity analysis confirms its stability. Finally, management suggestions was provided, this study contributes to the literature on public opinion risk assessment and provides implications for practice.

Heterogeneity in how women value risk-stratified breast screening

PURPOSE

Risk-stratified screening has potential to improve the cost effectiveness of national breast cancer screening programs. This study aimed to inform a socially acceptable and equitable implementation framework by determining what influences a woman's decision to accept a personalized breast cancer risk assessment and what the relative impact of these key determinants is.

METHODS

Multicriteria decision analysis was used to elicit the relative weights for 8 criteria that women reported influenced their decision. Preference heterogeneity was explored through cluster analysis.

RESULTS

The 2 criteria valued most by the 347 participants related to program access, "Mode of invitation" and "Testing process". Both criteria significantly influenced participation (P < .001). A total of 73% preferred communication by letter/online. Almost all women preferred a multidisease risk assessment with potential for a familial high-risk result. Four preference-based subgroups were identified. Membership to the largest subgroup was predicted by lower educational attainment, and women in this subgroup were concerned with program access. Higher relative perceived breast cancer risk predicted membership to the smallest subgroup that was focused on test parameters, namely "Scope of test" and "Test specificity".

CONCLUSION

Overall, Australian women would accept a personalized multidisease risk assessment, but when aligning with their preferences, it will necessitate a focus on program access and the development of online communication frameworks.

Assessing Patient Preferences in Rare Diseases: Direct Preference Elicitation in the Rare Chronic Kidney Disease, Immunoglobulin A Nephropathy

Background

Patient preference information is increasingly being used to inform decision making; however, further work is required to support the collection of preference information in rare diseases. This study illustrates the use of direct preference elicitation methods to collect preference data from small samples in the context of early decision making to inform the development of a product for the treatment of immunoglobulin A nephropathy.

Method

An interview-based swing weighting approach was used to elicit preferences from 40 patients in the US and China. Attributes were identified through a background review, expert engagement and patient focus groups. Participants completed a series of tasks that involved ranking, rating and scoring improvements in the attributes to obtain attribute swing weights and partial value functions. The preference results were then incorporated into a benefit-risk assessment simulation tool.

Results

Participants placed the greatest value on avoiding end-stage renal/kidney disease. Similar weight was given to short-term quality-of-life improvements and avoiding infections. Treatment burden (number of vaccinations) received the least weight. Heterogeneity in preferences was also observed. Consistency tests did not identify statistically significant variation in preferences, and qualitative data suggested that the elicitation exercise was sensitive to participants’ interpretation of attributes and that participants were able to express their preferences.

Conclusion

Direct preference elicitation methods can be used to collect preference data from small samples. Further work should continue to test the validity of the estimate generated by such methods.

Supplementary Information

The online version contains supplementary material available at 10.1007/s40271-021-00521-3.

A Systematic Review of Discrete Choice Experiments in Oncology Treatments

BACKGROUND

As the number and type of cancer treatments available rises and patients live with the consequences of their disease and treatments for longer, understanding preferences for cancer care can help inform decisions about optimal treatment development, access, and care provision. Discrete choice experiments (DCEs) are commonly used as a tool to elicit stakeholder preferences; however, their implementation in oncology may be challenging if burdensome trade-offs (e.g. length of life versus quality of life) are involved and/or target populations are small.

OBJECTIVES

The aim of this review was to characterise DCEs relating to cancer treatments that were conducted between 1990 and March 2020.

DATA SOURCES

EMBASE, MEDLINE, and the Cochrane Database of Systematic Reviews were searched for relevant studies.

STUDY ELIGIBILITY CRITERIA

Studies were included if they implemented a DCE and reported outcomes of interest (i.e. quantitative outputs on participants' preferences for cancer treatments), but were excluded if they were not focused on pharmacological, radiological or surgical treatments (e.g. cancer screening or counselling services), were non-English, or were a secondary analysis of an included study.

ANALYSIS METHODS

Analysis followed a narrative synthesis, and quantitative data were summarised using descriptive statistics, including rankings of attribute importance.

RESULT

Seventy-nine studies were included in the review. The number of published DCEs relating to oncology grew over the review period. Studies were conducted in a range of indications (n = 19), most commonly breast (n =10, 13%) and prostate (n = 9, 11%) cancer, and most studies elicited preferences of patients (n = 59, 75%). Across reviewed studies, survival attributes were commonly ranked as most important, with overall survival (OS) and progression-free survival (PFS) ranked most important in 58% and 28% of models, respectively. Preferences varied between stakeholder groups, with patients and clinicians placing greater importance on survival outcomes, and general population samples valuing health-related quality of life (HRQoL). Despite the emphasis of guidelines on the importance of using qualitative research to inform attribute selection and DCE designs, reporting on instrument development was mixed.

LIMITATIONS

No formal assessment of bias was conducted, with the scope of the paper instead providing a descriptive characterisation. The review only included DCEs relating to cancer treatments, and no insight is provided into other health technologies such as cancer screening. Only DCEs were included.

CONCLUSIONS AND IMPLICATIONS

Although there was variation in attribute importance between responder types, survival attributes were consistently ranked as important by both patients and clinicians. Observed challenges included the risk of attribute dominance for survival outcomes, limited sample sizes in some indications, and a lack of reporting about instrument development processes.

PROTOCOL REGISTRATION

PROSPERO 2020 CRD42020184232.

Similarity Analysis of Methods for Objective Determination of Weights in Multi-Criteria Decision Support Systems

Decision support systems (DSS) are currently developing rapidly and are increasingly used in various fields. More often, those systems are inseparable from information-based systems and computer systems. Therefore, from a methodical point of view, the algorithms implemented in the DSS play a critical role. In this aspect, multi-criteria decision support (MCDA) methods are widely used. As research progresses, many MCDA methods and algorithms for the objective identification of the significance of individual criteria of the MCDA models were developed. In this paper, an analysis of available objective methods for criteria weighting is presented. Additionally, the authors presented the implementation of the system that provides easy and accessible weight calculations for any decision matrix with the possibility of comparing results of different weighting methods. The results of weighting methods were compared using carefully selected similarity coefficients to emphasise the correlation of the resulting weights. The performed research shows that every method should provide distinctive weights considering input data, emphasising the importance of choosing the correct method for a given multi-criteria decision support model and DSS.

Evaluating an integrated care pathway for frail elderly patients in Norway using multi-criteria decision analysis

Background

To provide value-based care for patients with multi-morbidity, innovative integrated care programmes and comprehensive evaluations of such programmes are required. In Norway, a new programme called “Holistic Continuity of Patient Care” (HCPC) addresses the issue of multi-morbidity by providing integrated care within learning networks for frail elderly patients who receive municipal home care services or a short-term stay in a nursing home. This study conducts a multi-criteria decision analysis (MCDA) to evaluate whether the HCPC programme performs better on a large set of outcomes corresponding to the ‘triple aim’ compared to usual care.

Methods

Prospective longitudinal survey data were collected at baseline and follow-up after 6-months. The assessment of HCPC was implemented by a novel MCDA framework. The relative weights of importance of the outcomes used in the MCDA were obtained from a discrete choice experiment among five different groups of stakeholders. The performance score was estimated using a quasi-experimental design and linear mixed methods. Performance scores were standardized and multiplied by their weights of importance to obtain the overall MCDA value by stakeholder group.

Results

At baseline in the HCPC and usual care groups, respectively, 120 and 89 patients responded, of whom 87 and 41 responded at follow-up. The average age at baseline was 80.0 years for HCPC and 83.6 for usual care. Matching reduced the standardized differences between the groups for patient background characteristics and outcome variables. The MCDA results indicated that HCPC was preferred to usual care irrespective of stakeholders. The better performance of HCPC was mostly driven by improvements in enjoyment of life, psychological well-being, and social relationships and participation. Results were consistent with sensitivity analyses using Monte Carlo simulation.

Conclusion

Frail elderly with multi-morbidity represent complex health problems at large costs for society in terms of health- and social care. This study is a novel contribution to assessing and understanding HCPC programme performance respecting the multi-dimensionality of desired outcomes. Integrated care programmes like HCPC may improve well-being of patients, be cost-saving, and contribute to the pursuit of evidence based gradual reforms in the care of frail elderly.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-021-06805-6.

Patient Preferences for Lung Cancer Treatments: A Study Protocol for a Preference Survey Using Discrete Choice Experiment and Swing Weighting

Background: Advanced treatment options for non-small cell lung cancer (NSCLC) consist of immunotherapy, chemotherapy, or a combination of both. Decisions surrounding NSCLC can be considered as preference-sensitive because multiple treatments exist that vary in terms of mode of administration, treatment schedules, and benefit–risk profiles. As part of the IMI PREFER project, we developed a protocol for an online preference survey for NSCLC patients exploring differences in preferences according to patient characteristics (preference heterogeneity). Moreover, this study will evaluate and compare the use of two different preference elicitation methods, the discrete choice experiment (DCE) and the swing weighting (SW) task. Finally, the study explores how demographic (i.e., age, gender, and educational level) and clinical (i.e., cancer stage and line of treatment) information, health literacy, health locus of control, and quality of life may influence or explain patient preferences and the usefulness of a digital interactive tool in providing information on preference elicitation tasks according to patients.

Methods: An online survey will be implemented with the aim to recruit 510 NSCLC patients in Belgium and Italy. Participants will be randomized 50:50 to first receive either the DCE or the SW. The survey will also collect information on participants' disease-related status, health locus of control, health literacy, quality of life, and perception of the educational tool.

Discussion: This protocol outlines methodological and practical steps to quantitatively elicit and study patient preferences for NSCLC treatment alternatives. Results from this study will increase the understanding of which treatment aspects are most valued by NSCLC patients to inform decision-making in drug development, regulatory approval, and reimbursement. Methodologically, the comparison between the DCE and the SW task will be valuable to gain information on how these preference methods perform against each other in eliciting patient preferences. Overall, this protocol may assist researchers, drug developers, and decision-makers in designing quantitative patient preferences into decision-making along the medical product life cycle.

Patient Preferences for Multiple Myeloma Treatments: A Multinational Qualitative Study

Background: Investigational and marketed drugs for the treatment of multiple myeloma (MM) are associated with a range of characteristics and uncertainties regarding long term side-effects and efficacy. This raises questions about what matters most to patients living with this disease. This study aimed to understand which characteristics MM patients find most important, and hence should be included as attributes and levels in a subsequent quantitative preference survey among MM patients. Methods: This qualitative study involved: (i) a scoping literature review, (ii) discussions with MM patients (n = 24) in Belgium, Finland, Romania, and Spain using Nominal Group Technique, (iii) a qualitative thematic analysis including multi-stakeholder discussions. Results: MM patients voiced significant expectations and hopes that treatments would extend their lives and reduce their cancer signs and symptoms. Participants however raised concerns about life-threatening side-effects that could cause permanent organ damage. Bone fractures and debilitating neuropathic effects (such as chronic tingling sensations) were highlighted as major issues reducing patients' independence and mobility. Patients discussed the negative impact of the following symptoms and side-effects on their daily activities: thinking problems, increased susceptibility to infections, reduced energy, pain, emotional problems, and vision problems. MM patients were concerned with uncertainties regarding the durability of positive treatment outcomes, and the cause, severity, and duration of their symptoms and side-effects. Patients feared short-term positive treatment responses complicated by permanent, severe side-effects and symptoms. Conclusions: This study gained an in-depth understanding of the treatment and disease-related characteristics and types of attribute levels (severity, duration) that are most important to MM patients. Results from this study argue in favor of MM drug development and individual treatment decision-making that focuses not only on extending patients' lives but also on addressing those symptoms and side-effects that significantly impact MM patients' quality of life. This study underscores a need for transparent communication toward MM patients about MM treatment outcomes and uncertainties regarding their long-term efficacy and safety. Finally, this study may help drug developers and decision-makers understand which treatment outcomes and uncertainties are most important to MM patients and therefore should be incorporated in MM drug development, evaluation, and clinical practice.

A Multicriteria Decision Analysis Comparing Pharmacotherapy for Chronic Neuropathic Pain, Including Cannabinoids and Cannabis-Based Medical Products

Background:

Pharmacological management of chronic neuropathic pain (CNP) still represents a major clinical challenge. Collective harnessing of both the scientific evidence base and clinical experience (of clinicians and patients) can play a key role in informing treatment pathways and contribute to the debate on specific treatments (e.g., cannabinoids). A group of expert clinicians (pain specialists and psychiatrists), scientists, and patient representatives convened to assess the relative benefit–safety balance of 12 pharmacological treatments, including orally administered cannabinoids/cannabis-based medicinal products, for the treatment of CNP in adults.

Methods:

A decision conference provided the process of creating a multicriteria decision analysis (MCDA) model, in which the group collectively scored the drugs on 17 effect criteria relevant to benefits and safety and then weighted the criteria for their clinical relevance.

Findings:

Cannabis-based medicinal products consisting of tetrahydrocannabinol/cannabidiol (THC/CBD), in a 1:1 ratio, achieved the highest overall score, 79 (out of 100), followed by CBD dominant at 75, then THC dominant at 72. Duloxetine and the gabapentinoids scored in the 60s, amitriptyline, tramadol, and ibuprofen in the 50s, methadone and oxycodone in the 40s, and morphine and fentanyl in the 30s. Sensitivity analyses showed that even if the pain reduction and quality-of-life scores for THC/CBD and THC are halved, their benefit–safety balances remain better than those of the noncannabinoid drugs.

Interpretation:

The benefit–safety profiles for cannabinoids were higher than for other commonly used medications for CNP largely because they contribute more to quality of life and have a more favorable side effect profile. The results also reflect the shortcomings of alternative pharmacological treatments with respect to safety and mitigation of neuropathic pain symptoms. Further high-quality clinical trials and systematic comprehensive capture of clinical experience with cannabinoids is warranted. These results demonstrate once again the complexity and multimodal mechanisms underlying the clinical experience and impact of chronic pain.

Patient Preferences to Assess Value IN Gene Therapies: Protocol Development for the PAVING Study in Hemophilia

Introduction: Gene therapies are innovative therapies that are increasingly being developed. However, health technology assessment (HTA) and payer decision making on these therapies is impeded by uncertainties, especially regarding long-term outcomes. Through measuring patient preferences regarding gene therapies, the importance of unique elements that go beyond health gain can be quantified and inform value assessments. We designed a study, namely the Patient preferences to Assess Value IN Gene therapies (PAVING) study, that can inform HTA and payers by investigating trade-offs that adult Belgian hemophilia A and B patients are willing to make when asked to choose between a standard of care and gene therapy.

Methods and Analysis: An eight-step approach was taken to establish the protocol for this study: (1) stated preference method selection, (2) initial attributes identification, (3) stakeholder (HTA and payer) needs identification, (4) patient relevant attributes and information needs identification, (5) level identification and choice task construction, (6) educational tool design, (7) survey integration, and (8) piloting and pretesting. In the end, a threshold technique survey was designed using the attributes “Annual bleeding rate,” “Chance to stop prophylaxis,” “Time that side effects have been studied,” and “Quality of Life.”

Ethics and Dissemination: The Medical Ethics Committee of UZ KU Leuven/Research approved the study. Results from the study will be presented to stakeholders and patients at conferences and in peer-reviewed journals. We hope that results from the PAVING study can inform decision makers on the acceptability of uncertainties and the value of gene therapies to patients.

Using Eye-Tracking Technology with Older People in Memory Clinics to Investigate the Impact of Mild Cognitive Impairment on Choices for EQ-5D-5L Health States Preferences

BACKGROUND

Population ageing is a phenomenon taking place in almost every global region. Current estimates indicate that 10-20% of older people in developed countries have mild cognitive impairment (MCI), with these percentages predicted to rise markedly by 2050.

OBJECTIVE

Our objective was to apply eye-tracking technology to investigate the information processes adopted by older people with and without MCI in determining preferences for health states in the five-level EuroQol-5 Dimensions (EQ-5D-5L) instrument.

METHODS

Older people (aged ≥ 65 years; including both patients and family carers) attending outpatient memory clinics in Southern Adelaide between July 2017 and June 2018, competent to read and converse in English and with a Mini-Mental State Examination (MMSE) cognition score of ≥ 19 were invited to participate. In total, 52 people met the inclusion criteria, of whom 20 (38%) provided informed consent and fully participated. Participants were categorised into two subgroups (each n = 10) for comparison based upon established MMSE cognition thresholds (19-23, lower MMSE indicative of MCI; ≥ 24, higher MMSE indicative of good cognition). A discrete-choice experiment (DCE) comprising a series of pairwise choices between alternative EQ-5D-5L health states of varying survival duration with differential levels of task complexity (approximated by the degree of attribute level overlap in each choice), was administered as a face-to-face interview with the participant wearing an eye-tracking device.

RESULTS

Attribute non-attendance (ANA) was higher for the lower MMSE subgroup than for the higher MMSE subgroup, although these differences were generally not statistically significant. ANA remained relatively low and consistent for participants with good cognition regardless of task complexity. In contrast, ANA increased notably in participants exhibiting MCI, increasing from 10% on average per participant in the lower MMSE subgroup with five attribute level overlap to 23% on average per participant in the lower MMSE subgroup with zero attribute level overlap.

CONCLUSIONS

This exploratory study provided important insights into the information processes adopted by older people with varying levels of cognitive functioning when choosing between alternative EQ-5D-5L health states of varying survival duration and specifically the relationships between cognitive capacity, task complexity and the extent of ANA. Recent advances in econometric modelling of health state valuation data have demonstrated the added value of capturing ANA information as this can be accounted for in the DCE data analysis, thereby improving the precision of model estimates. Eye-tracking technology can usefully inform the design, conduct and econometric modelling of DCEs, driving the inclusion of this rapidly growing population traditionally excluded from preference-elicitation studies of this nature.

Structured benefit-risk evaluation for medicinal products: review of quantitative benefit-risk assessment findings in the literature

A favorable benefit–risk profile remains an essential requirement for marketing authorization of medicinal drugs and devices. Furthermore, prior subjective, implicit and inconsistent ad hoc benefit–risk assessment methods have rightly evolved towards more systematic, explicit or “structured” approaches. Contemporary structured benefit–risk evaluation aims at providing an objective assessment of the benefit–risk profile of medicinal products and a higher transparency for decision making purposes. The use of a descriptive framework should be the preferred starting point for a structured benefit–risk assessment. In support of more precise assessments, quantitative and semi-quantitative methodologies have been developed and utilized to complement descriptive or qualitative frameworks in order to facilitate the structured evaluation of the benefit–risk profile of medicinal products. In addition, quantitative structured benefit–risk analysis allows integration of patient preference data. Collecting patient perspectives throughout the medical product development process has become increasingly important and key to the regulatory decision-making process. Both industry and regulatory authorities increasingly rely on descriptive structured benefit–risk evaluation and frameworks in drug, vaccine and device evaluation and comparison. Although varied qualitative methods are more commonplace, quantitative approaches have recently been emphasized. However, it is unclear how frequently these quantitative frameworks have been used by pharmaceutical companies to support submission dossiers for drug approvals or to respond to the health authorities’ requests. The objective of this study has been to identify and review, for the first time, currently available, published, structured, quantitative benefit–risk evaluations which may have informed health care professionals and/or payor as well as contributed to decision making purposes in the regulatory setting for drug, vaccine and/or device approval.

Use of Patient Preference Information in Benefit-Risk Assessment, Health Technology Assessment, and Pricing and Reimbursement Decisions: A Systematic Literature Review of Attempts and Initiatives

Objectives: Inclusion of patient preference (PP) data in decision making has been largely discussed in recent years. Healthcare decision makers—regulatory and health technology assessment (HTA)—are more and more conscious of the need for a patient-centered approach to decide on optimal allocation of scarce money, time, and technological resources. This literature review aims to examine the use of and recommendations for the integration of PP in decision making.

Methods: A literature search was conducted through PubMed/Medline in May 2019 to identify publications on PP studies used to inform benefit–risk assessments (BRAs) and HTAs and patient-centered projects and guidelines related to the inclusion of PPs in health policy decision making. After title and abstract screening and full-text review, selected publications were analyzed to retrieve data related to the collection, use, and/or submission of PPs informing BRA or HTA as well as attempts and initiatives in recommendations for PPs integration in decision-making processes.

Results: Forty-nine articles were included: 24 attempts and pilot project discussions and 25 PP elicitation studies. Quantitative approaches, particularly discrete choice experiments, were the most used (24 quantitative elicitation studies and 1 qualitative study). The objective of assessing PPs was to prioritize outcome-specific information, to value important treatment characteristics, to provide patient-focused benefit–risk trade-offs, and to appraise the patients' willingness to pay for new technologies. Moreover, attempts and pilot projects to integrate PPs in BRAs and HTAs were identified at the European level and across countries, but no clear recommendations have been issued yet. No less than seven public and/or private initiatives have been undertaken by governmental agencies and independent organizations to set guidance targeting improvement of patients' involvement in decision making.

Conclusion: Despite the initiatives undertaken, the pace of progress remains slow. The use of PPs remains poorly implemented, and evidence of proper use of these data in decision making is lacking. Guidelines and recommendations formalizing the purpose of collecting PPs, what methodology should be adopted and how, and who should be responsible for generating these data throughout the decision-making processes are needed to improve and empower integration of PPs in BRA and HTA.

Reporting Quality of Marginal Rates of Substitution in Discrete Choice Experiments That Elicit Patient Preferences

BACKGROUND

Discrete choice experiments (DCEs) are commonly used to elicit patient preferences as marginal rates of substitution (MRSs) between treatment or health service attributes. Because these studies are increasing in importance, it is vital that uncertainty around MRS estimates is reported.

OBJECTIVE

To review recently published DCE studies that elicit patient preferences in relation to MRS reporting and to explore the accuracy of using other reported information to estimate the uncertainty of the MRSs.

METHODS

A systematic literature review of DCEs conducted with patients between 2014 and July 2019 was performed. The number of studies reporting coefficients, MRSs, standard errors (SEs), and confidence intervals was recorded. If all information was reported, studies were included in an analysis to determine the impact of estimating the SEs of MRSs using coefficients and assuming zero covariance, to determine the impact of this assumption.

RESULTS

Two hundred and thirty-two patient DCEs were identified in the review; 34.1% (n = 79) reported 1 or more MRS and, of these, only 62.0% (n = 49) provided an estimate of the uncertainty. Of these studies, 16 contained enough information for inclusion in the analysis, providing 116 datapoints. Actual SEs were smaller than estimated SEs in 75.0% of cases (n = 87), and estimated SEs were within 25% of the actual SE in 59.5% of cases (n = 69).

CONCLUSION

Uncertainty of MRS estimates is unreported in a substantial proportion of recently published DCE studies. Estimating the SE of a MRS by solely using the SEs of the utility coefficients is likely to lead to biased estimates of the precision of patient trade-offs.

Quality of life and patient preferences among Danish patients with ulcerative colitis - results from a survey study

Objective: To explore health-related quality of life (HRQoL) and assess preferences for medical treatment attributes to obtain information of the relative importance of the different attributes in a Danish population with ulcerative colitis (UC).Methods: We used data from an online survey collected in March 2018 among people with self-reported UC. A total of 302 eligible respondents answered the HRQoL questionnaires (EuroQol-5 Dimensions (EQ-5D-5L) and the Short Inflammatory Bowel Disease Questionnaire (SIBDQ)), and 212 also completed the discrete choice experiment (DCE). The probability of choosing an alternative from a number of choices in the DCE was estimated using a conditional logit model.Results: The respondents had an average SIBDQ score of 4.5 and an HRQoL score of 0.77, applying the EQ-5D-5L questionnaire. HRQoL correlated with disease severity, and the respondents had lower HRQoL than did a gender- and age-matched subset of the Danish population. The most important medical treatment attribute was efficacy within eight weeks. Additionally, respondents stated a preference for avoiding taking steroids, for fast onset of effect and for oral formulations.Conclusions: HRQoL correlates with disease severity, and patients with UC have lower HRQoL than the general population. The most important treatment attribute was efficacy, but patients also would like to avoid steroids, value fast onset of effect and prefer oral formulations.

How to support the application of multiple criteria decision analysis? Let us start with a comprehensive taxonomy

Decision making is a complex task that involves a multitude of perspectives, constraints, and variables. Multiple Criteria Decision Analysis (MCDA) is a process that has been used for several decades to support decision making. It includes a series of steps that systematically help Decision Maker(s) (DM(s)) and stakeholders in structuring a decision making problem, identifying their preferences, and building a decision recommendation consistent with those preferences. Over the last decades, many studies have demonstrated the conduct of the MCDA process and how to select an MCDA method. Until now, there has not been a review of these studies, nor a proposal of a unified and comprehensive high-level representation of the MCDA process characteristics (i.e., features), which is the goal of this paper. We introduce a review of the research that defines how to conduct the MCDA process, compares MCDA methods, and presents Decision Support Systems (DSSs) to recommend a relevant MCDA method or a subset of methods. We then synthesize this research into a taxonomy of characteristics of the MCDA process, grouped into three main phases, (i) problem formulation, (ii) construction of the decision recommendation, and (iii) qualitative features and technical support. Each of these phases includes a subset of the 10 characteristics that helps the analyst implementing the MCDA process, while also being aware of the implication of these choices at each step. By showing how decision making can be split into manageable and justifiable steps, we reduce the risk of overwhelming the analyst, as well as the DMs/stakeholders during the MCDA process. A questioning strategy is also proposed to demonstrate how to apply the taxonomy to map MCDA methods and select the most relevant one(s) using real case studies. Additionally, we show how the DSSs for MCDA method recommendation can be grouped into three main clusters. This proposal can enhance a traceable and categorizable development of such systems.

Personalizing Medical Treatment Decisions: Integrating Meta-analytic Treatment Comparisons with Patient-Specific Risks and Preferences

Background. Network meta-analyses (NMAs) that compare treatments for a given condition allow physicians to identify which treatments have higher or lower probabilities of reducing the risks of disease complications or increasing the risks of treatment side effects. Translating these data into personalized treatment plans requires integration of NMA data with patient-specific pretreatment risk estimates and preferences regarding treatment objectives and acceptable risks. Methods. We introduce a modeling framework to integrate data probabilistically from NMAs with data on individualized patient risk estimates for disease outcomes, treatment preferences (such as willingness to incur greater side effects for increased life expectancy), and risk preferences. We illustrate the modeling framework by creating personalized plans for antipsychotic drug treatment and evaluating their effectiveness and cost-effectiveness. Results. Compared with treating all patients with the drug that yields the greatest quality-adjusted life-years (QALYs) on average (amisulpride), personalizing the selection of antipsychotic drugs for schizophrenia patients over the next 5 years would be expected to yield 0.33 QALYs (95% credible interval [crI]: 0.30-0.37) per patient at an incremental cost of $4849/QALY gained (95% crI: dominant-$12,357), versus 0.29 and 0.04 QALYs per patient when accounting for only risks or preferences, respectively, but not both. Limitations. The analysis uses a linear, additive utility function to reflect patient treatment preferences and does not consider potential variations in patient time discounting. Conclusions. Our modeling framework rigorously computes what physicians normally have to do mentally. By integrating 3 key components of personalized medicine-evidence on efficacy, patient risks, and patient preferences-the modeling framework can provide personalized treatment decisions to improve patient health outcomes.

Opportunities and challenges for the inclusion of patient preferences in the medical product life cycle: a systematic review

BACKGROUND

The inclusion of patient preferences (PP) in the medical product life cycle is a topic of growing interest to stakeholders such as academics, Health Technology Assessment (HTA) bodies, reimbursement agencies, industry, patients, physicians and regulators. This review aimed to understand the potential roles, reasons for using PP and the expectations, concerns and requirements associated with PP in industry processes, regulatory benefit-risk assessment (BRA) and marketing authorization (MA), and HTA and reimbursement decision-making.

METHODS

A systematic review of peer-reviewed and grey literature published between January 2011 and March 2018 was performed. Consulted databases were EconLit, Embase, Guidelines International Network, PsycINFO and PubMed. A two-step strategy was used to select literature. Literature was analyzed using NVivo (QSR international).

RESULTS

From 1015 initially identified documents, 72 were included. Most were written from an academic perspective (61%) and focused on PP in BRA/MA and/or HTA/reimbursement (73%). Using PP to improve understanding of patients' valuations of treatment outcomes, patients' benefit-risk trade-offs and preference heterogeneity were roles identified in all three decision-making contexts. Reasons for using PP relate to the unique insights and position of patients and the positive effect of including PP on the quality of the decision-making process. Concerns shared across decision-making contexts included methodological questions concerning the validity, reliability and cognitive burden of preference methods. In order to use PP, general, operational and quality requirements were identified, including recognition of the importance of PP and ensuring patient understanding in PP studies.

CONCLUSIONS

Despite the array of opportunities and added value of using PP throughout the different steps of the MPLC identified in this review, their inclusion in decision-making is hampered by methodological challenges and lack of specific guidance on how to tackle these challenges when undertaking PP studies. To support the development of such guidance, more best practice PP studies and PP studies investigating the methodological issues identified in this review are critically needed.

Factors and Situations Affecting the Value of Patient Preference Studies: Semi-Structured Interviews in Europe and the US

Objectives: Patient preference information (PPI) is gaining recognition among the pharmaceutical industry, regulatory authorities, and health technology assessment (HTA) bodies/payers for use in assessments and decision-making along the medical product lifecycle (MPLC). This study aimed to identify factors and situations that influence the value of patient preference studies (PPS) in decision-making along the MPLC according to different stakeholders.

Methods: Semi-structured interviews (n = 143) were conducted with six different stakeholder groups (physicians, academics, industry representatives, regulators, HTA/payer representatives, and a combined group of patients, caregivers, and patient representatives) from seven European countries (the United Kingdom, Sweden, Italy, Romania, Germany, France, and the Netherlands) and the United States. Framework analysis was performed using NVivo 11 software.

Results: Fifteen factors affecting the value of PPS in the MPLC were identified. These are related to: study organization (expertise, financial resources, study duration, ethics and good practices, patient centeredness), study design (examining patient and/or other preferences, ensuring representativeness, matching method to research question, matching method to MPLC stage, validity and reliability, cognitive burden, patient education, attribute development), and study conduct (patients’ ability/willingness to participate and preference heterogeneity). Three types of situations affecting the use of PPS results were identified (stakeholder acceptance, market situations, and clinical situations).

Conclusion: The factors and situation types affecting the value of PPS, as identified in this study, need to be considered when designing and conducting PPS in order to promote the integration of PPI into decision-making along the MPLC.

From Individual to Population Preferences: Comparison of Discrete Choice and Dirichlet Models for Treatment Benefit-Risk Tradeoffs

Introduction. The Dirichlet distribution has been proposed for representing preference heterogeneity, but there is limited evidence on its suitability for modeling population preferences on treatment benefits and risks. Methods. We conducted a simulation study to compare how the Dirichlet and standard discrete choice models (multinomial logit [MNL] and mixed logit [MXL]) differ in their convergence to stable estimates of population benefit-risk preferences. The source data consisted of individual-level tradeoffs from an existing 3-attribute patient preference study (N = 560). The Dirichlet population model was fit directly to the attribute weights in the source data. The MNL and MXL population models were fit to the outcomes of a simulated discrete choice experiment in the same sample of 560 patients. Convergence to the parameter values of the Dirichlet and MNL population models was assessed with sample sizes ranging from 20 to 500 (100 simulations per sample size). Model variability was also assessed with coefficient P values. Results. Population preference estimates of all models were very close to the sample mean, and the MNL and MXL models had good fit (McFadden's adjusted R² = 0.12 and 0.13). The Dirichlet model converged reliably to within 0.05 distance of the population preference estimates with a sample size of 100, where the MNL model required a sample size of 240 for this. The MNL model produced consistently significant coefficient estimates with sample sizes of 100 and higher. Conclusion. The Dirichlet model is likely to have smaller sample size requirements than standard discrete choice models in modeling population preferences for treatment benefit-risk tradeoffs and is a useful addition to health preference analyst's toolbox.

A novel measure of drug benefit-risk assessment based on Scale Loss Score

Quantitative methods have been proposed to assess and compare the benefit-risk balance of treatments. Among them, multicriteria decision analysis (MCDA) is a popular decision tool as it permits to summarise the benefits and the risks of a drug in a single utility score, accounting for the preferences of the decision-makers. However, the utility score is often derived using a linear model which might lead to counter-intuitive conclusions; for example, drugs with no benefit or extreme risk could be recommended. Moreover, it assumes that the relative importance of benefits against risks is constant for all levels of benefit or risk, which might not hold for all drugs. We propose Scale Loss Score (SLoS) as a new tool for the benefit-risk assessment, which offers the same advantages as the linear multicriteria decision analysis utility score but has, in addition, desirable properties permitting to avoid recommendations of non-effective or extremely unsafe treatments, and to tolerate larger increases in risk for a given increase in benefit when the amount of benefit is small than when it is high. We present an application to a real case study on telithromycin in Community Acquired Pneumonia and Acute Bacterial Sinusitis, and we investigated the patterns of behaviour of Scale Loss Score, as compared to the linear multicriteria decision analysis, in a comprehensive simulation study.

Multi-criteria decision analysis for health technology assessment: addressing methodological challenges to improve the state of the art

BACKGROUND

Multi-criteria decision analysis (MCDA) concepts, models and tools have been used increasingly in health technology assessment (HTA), with several studies pointing out practical and theoretical issues related to its use. This study provides a critical review of published studies on MCDA in the context of HTA by assessing their methodological quality and summarising methodological challenges.

METHODS

A systematic review was conducted to identify studies discussing, developing or reviewing the use of MCDA in HTA using aggregation approaches. Studies were classified according to publication time and type, country of study, technology type and study type. The PROACTIVE-S approach was constructed and used to analyse methodological quality. Challenges and limitations reported in eligible studies were collected and summarised; this was followed by a critical discussion on research requirements to address the identified challenges.

RESULTS

129 journal articles were eligible for review, 56% of which were published in 2015-2017; 42% focused on pharmaceuticals; 36, 26 and 18% reported model applications, issues regarding MCDA implementation analyses, and proposing frameworks, respectively. Poor compliance with good methodological practice (< 25% complying studies) was found regarding behavioural analyses, discussion of model assumptions and uncertainties, modelling of value functions, and dealing with judgment inconsistencies. The five most reported challenges related to evidence and data synthesis; value system differences and participant selection issues; participant difficulties; methodological complexity and resource balance; and criteria and attributes modelling. A critical discussion on ways to address these challenges ensues.

DISCUSSION

Results highlight the need for advancement in robust methodologies, procedures and tools to improve methodological quality of MCDA in HTA studies. Research pathways include developing new model features, good practice guidelines, technologies to enable participation and behavioural research.

Quantifying Preferences in Drug Benefit-Risk Decisions

Benefit-risk assessment is used in various phases along the drug lifecycle, such as marketing authorization and surveillance, health technology assessment (HTA), and clinical decisions, to understand whether, and for which patients, a drug has a favorable or more valuable profile with reference to one or more comparators. Such assessments are inherently preference-based as several clinical and nonclinical outcomes of varying importance might act as evaluation criteria, and decision makers must establish acceptable trade-offs between these outcomes. Different healthcare stakeholder perspectives, such as those from patients and healthcare professionals, are key for informing benefit-risk trade-offs. However, the degree to which such preferences inform the decision is often unclear as formal preference-based evaluation frameworks are generally not used for regulatory decisions, and, if used, rarely communicated in HTA decisions. We argue that for better decisions, as well as for reasons of transparency, preferences in benefit-risk decisions should more often be quantified and communicated explicitly.

A Bayesian approach for individual-level drug benefit-risk assessment

In existing benefit-risk assessment (BRA) methods, benefit and risk criteria are usually identified and defined separately based on aggregated clinical data and therefore ignore the individual-level differences as well as the association among the criteria. We proposed a Bayesian multicriteria decision-making method for BRA of drugs using individual-level data. We used a multidimensional latent trait model to account for the heterogeneity of treatment effects with latent variables introducing the dependencies among outcomes. We then applied the stochastic multicriteria acceptability analysis approach for BRA incorporating imprecise and heterogeneous patient preference information. We adopted an efficient Markov chain Monte Carlo algorithm when implementing the proposed method. We applied our method to a case study to illustrate how individual-level benefit-risk profiles could inform decision-making.

Personalizing Second-Line Type 2 Diabetes Treatment Selection: Combining Network Meta-analysis, Individualized Risk, and Patient Preferences for Unified Decision Support

BACKGROUND

Personalizing medical treatment often requires practitioners to compare multiple treatment options, assess a patient's unique risk and benefit from each option, and elicit a patient's preferences around treatment. We integrated these 3 considerations into a decision-modeling framework for the selection of second-line glycemic therapy for type 2 diabetes.

METHODS

Based on multicriteria decision analysis, we developed a unified treatment decision support tool accounting for 3 factors: patient preferences, disease outcomes, and medication efficacy and safety profiles. By standardizing and multiplying these 3 factors, we calculated the ranking score for each medication. This approach was applied to determining second-line glycemic therapy by integrating 1) treatment efficacy and side-effect data from a network meta-analysis of 301 randomized trials ( N = 219,277), 2) validated risk equations for type 2 diabetes complications, and 3) patient preferences around treatment (e.g., to avoid daily glucose testing). Data from participants with type 2 diabetes in the U.S. National Health and Nutrition Examination Survey (NHANES 2003-2014, N = 1107) were used to explore variations in treatment recommendations and associated quality-adjusted life-years given different patient features.

RESULTS

Patients at the highest microvascular disease risk had glucagon-like peptide 1 agonists or basal insulin recommended as top choices, whereas those wanting to avoid an injected medication or daily glucose testing had sodium-glucose linked transporter 2 or dipeptidyl peptidase 4 inhibitors commonly recommended, and those with major cost concerns had sulfonylureas commonly recommended. By converting from the most common sulfonylurea treatment to the model-recommended treatment, NHANES participants were expected to save an average of 0.036 quality-adjusted life-years per person (about a half month) from 10 years of treatment.

CONCLUSIONS

Models can help integrate meta-analytic treatment effect estimates with individualized risk calculations and preferences, to aid personalized treatment selection.

Factors and situations influencing the value of patient preference studies along the medical product lifecycle: a literature review

Industry, regulators, health technology assessment (HTA) bodies, and payers are exploring the use of patient preferences in their decision-making processes. In general, experience in conducting and assessing patient preference studies is limited. Here, we performed a systematic literature search and review to identify factors and situations influencing the value of patient preference studies, as well as applications throughout the medical product lifecyle. Factors and situations identified in 113 publications related to the organization, design, and conduct of studies, and to communication and use of results. Although current use of patient preferences is limited, we identified possible applications in discovery, clinical development, marketing authorization, HTA, and postmarketing phases.

Strengthening the evidence-base of integrated care for people with multi-morbidity in Europe using Multi-Criteria Decision Analysis (MCDA)

BACKGROUND

Evaluation of integrated care programmes for individuals with multi-morbidity requires a broader evaluation framework and a broader definition of added value than is common in cost-utility analysis. This is possible through the use of Multi-Criteria Decision Analysis (MCDA).

METHODS AND RESULTS

This paper presents the seven steps of an MCDA to evaluate 17 different integrated care programmes for individuals with multi-morbidity in 8 European countries participating in the 4-year, EU-funded SELFIE project. In step one, qualitative research was undertaken to better understand the decision-context of these programmes. The programmes faced decisions related to their sustainability in terms of reimbursement, continuation, extension, and/or wider implementation. In step two, a uniform set of decision criteria was defined in terms of outcomes measured across the 17 programmes: physical functioning, psychological well-being, social relationships and participation, enjoyment of life, resilience, person-centeredness, continuity of care, and total health and social care costs. These were supplemented by programme-type specific outcomes. Step three presents the quasi-experimental studies designed to measure the performance of the programmes on the decision criteria. Step four gives details of the methods (Discrete Choice Experiment, Swing Weighting) to determine the relative importance of the decision criteria among five stakeholder groups per country. An example in step five illustrates the value-based method of MCDA by which the performance of the programmes on each decision criterion is combined with the weight of the respective criterion to derive an overall value score. Step six describes how we deal with uncertainty and introduces the Conditional Multi-Attribute Acceptability Curve. Step seven addresses the interpretation of results in stakeholder workshops.

DISCUSSION

By discussing our solutions to the challenges involved in creating a uniform MCDA approach for the evaluation of different programmes, this paper provides guidance to future evaluations and stimulates debate on how to evaluate integrated care for multi-morbidity.