Preferences as fairness judgments: a critical review of normative frameworks of preference elicitation and development of an alternative based on constitutional economics

Preference elicitation is widely used within health economic evaluations to inform coverage decisions. However, coverage decisions involve questions of social justice and it is unclear what role empirical evidence about preferences can play here. This study reviews the prevalent normative frameworks for using population-based preference elicitation and the criticisms they face, and proposes an alternative based on constitutional economics. The frameworks reviewed include a supposedly value-neutral framework of preferences as predictors of choice, preference utilitarian frameworks that aim to maximize preference satisfaction, and substantive consequentialist frameworks that aim to maximize happiness, health, or capabilities. The proposed alternative implements the idea that indices of social value are tools for conflict resolution, rather than tools for maximization. Preference elicitation is used for validating values generated by multi-criteria decision analysis results within representative processes of stakeholder deliberation.

Eliciting preferences for cancer screening tests: Comparison of a discrete choice experiment and the threshold technique

OBJECTIVE

To compare results of three preference elicitation methods for a cancer screening test.

METHODS

Participants (undergraduate students) completed a discrete choice experiment (DCE) and a threshold technique (TT) task. Accuracy (false positives, false negatives), benefits (lives saved), and cost for a cancer screening test were used as attributes in the DCE and branching logic for the TT. Participants were also asked a direct elicitation question regarding a hypothetical screening test for breast (women) or prostate (men) cancer without mortality benefit. Correlations assessed the relationship between DCE and TT thresholds. Thresholds were standardized and ranked for both methods to compare. A logistic regression used the thresholds to predict results of the direct elicitation.

RESULTS

DCE and TT estimates were not meaningfully correlated (max ρ = 0.17). Participant rankings of attributes matched only 20% of the time (58/292). Neither method predicted preference for being screened (ps > 0.21).

CONCLUSIONS

The DCE and TT yielded different preference estimates (and rank orderings) for the same participant. Neither method predicted patients' desires for a screening test.

PRACTICE IMPLICATIONS

Clinicians, patients, policy makers, and researchers should be aware that patient preference results may be sensitive to the method of eliciting preferences.

Derivation of a UK preference-based value set for the Short Warwick-Edinburgh Mental Well-being Scale (SWEMWBS) to allow estimation of Mental Well-being Adjusted Life Years (MWALYs)

BACKGROUND

The Mental Well-being Adjusted Life Year (MWALY) is an alternative outcome measure to the quality-adjusted life year (QALY) in economic evaluations of interventions aimed at improving mental well-being. However, there is a lack of preference-based mental well-being instruments for capturing population mental well-being preferences.

OBJECTIVES

To derive a UK preference-based value set for the Short Warwick-Edinburgh Mental Well-being Scale (SWEMWBS).

METHODS

225 participants that were interviewed between December 2020 and August 2021 completed 10 composite time trade-off (C-TTO) and 10 discrete choice experiment (DCE) interviewer-administered exercises. Heteroskedastic Tobit and conditional logit models were used to model C-TTO and DCE responses respectively. The DCE utility values were rescaled to a C-TTO comparable scale through anchoring and mapping. An inverse variance weighting hybrid model (IVWHM) was used to derive weighted-average coefficients from the modelled C-TTO and DCE coefficients. Model performance was assessed using statistical diagnostics.

RESULTS

The valuation responses confirmed the feasibility and face validity of the C-TTO and DCE techniques. Apart from the main effects models, statistically significant associations were estimated between the predicted C-TTO value and participants' SWEMWBS scores, gender, ethnicities, education levels, and the interaction terms between age and useful feeling. The IVWHM was the most optimal model with the fewest logically inconsistent coefficients and the lowest pooled standard errors. The utility values generated by the rescaled DCE models and the IVWHM were generally higher than those of the C-TTO model. The predictive ability of the two DCE rescaling methods was similar according to the mean absolute deviation and root mean square deviation statistics.

CONCLUSIONS

This study has produced the first preference-based value set for a measure of mental well-being. The IVWHM provided a desirable blend of both C-TTO and DCE models. The value set derived by this hybrid approach can be used for cost-utility analyses of mental well-being interventions.

Eliciting preferences and respecting values: Why ask?

This essay explores the pitfalls and ambiguities in relying on preference elicitation to value health states, and it distinguishes preference elicitation, as a fallible method of measuring well-being, from public consultation, as an element of public deliberation. After distinguishing preference elicitation as a method of ascertaining opinions from preference elicitation as a method of measuring well-being, it points out that preferences depend on beliefs and the considerations speaking in favor of deferring to people's values do not carry over to deferring to their beliefs. Instead of valuing health states by their bearing on well-being, as measured by preferences, this essay argues for valuing health states by their bearing on activity limitations and suffering, as determined by public deliberation.

Optimizing Uptake of Long-Acting Injectable Pre-exposure Prophylaxis for HIV Prevention for Men Who Have Sex with Men

Pre-exposure prophylaxis (PrEP) is a highly effective HIV prevention tool. Long-acting injectable PrEP (LAI-PrEP) offers another opportunity to reduce HIV. However, how at-risk individuals will consider LAI-PrEP over other modes of administration is unclear. We conducted a discrete choice experiment on preferences for PrEP among a sample of N = 688 gay, bisexual, and other men who have sex with men (GBMSM). We analyzed preferences for mode of administration, side-effects, monetary cost, and time cost using a conditional logit model and predicted preference for PrEP options. LAI-PrEP was preferred, despite mode of administration being the least important PrEP attribute. Side-effects were the most important attribute influencing preferences for PrEP (44% of decision); costs were second-most-important (35% of decision). PrEP with no side-effects was the most important preference, followed by monthly out-of-pocket costs of $0. Practitioners and policymakers looking to increase PrEP uptake should keep costs low, communicate clearly about PrEP side-effects, and allow the use of patient-preferred modes of PrEP administration, including LAI-PrEP.

Quantifying public preferences for healthcare priorities in Taiwan through an integrated citizens jury and discrete choice experiment

Priority setting is a critical process for national healthcare systems that need to allocate limited resources across unlimited healthcare demands. In recent decades, health policymakers have identified the need to combine technical dimensions of priority setting with political dimensions relating to community values. A range of methods for engaging the public in priority setting has been developed, yet there is no consensus around the most effective methodology. A 2014 paper proposed the integration of two methods currently used for soliciting public preferences around health care services: i) an individual survey instrument, Discrete Choice Experiments (DCEs) and ii) Citizen Juries (CJs), a group-based model that incorporates education and deliberative dialogue. This pilot study is among the first to empirically test this integrated method to assess its value across two domains: does the CJ process alter participant preferences and are the consensus values of the CJ captured by the individualistic DCE? The two-part, mixed methods study was administered in Taipei, Taiwan in August of 2016. Twenty-seven participants completed a DCE as a baseline pre-test, ranking a set of attributes in terms of importance for future resource allocation under Taiwan's National Health Insurance System. Twenty of the participants next took part in the integrated CJ-DCE method, which consisted of education and facilitated dialogue through a CJ, followed by retaking the DCE survey. Participant preferences changed after undergoing the CJ process and these new, group-based preferences were reflected in the second DCE, meaning participants did not revert to their original individualistic preferences. The results of this study demonstrate that the integrated CJ-DCE method adds value in allowing an ethically communitarian set of values to be developed and captured via an individualistic methodology. Further testing is needed to investigate the reliability of our findings and how it may be implemented to maximize public acceptance.

Minimality and comparison of sets of multi-attribute vectors

In a decision-making problem, there is often some uncertainty regarding the user preferences. We assume a parameterised utility model, where in each scenario we have a utility function over alternatives, and where each scenario represents a possible user preference model consistent with the input preference information. With a set A of alternatives available to the decision-maker, we can consider the associated utility function, expressing, for each scenario, the maximum utility among the alternatives. We consider two main problems: firstly, finding a minimal subset of A that is equivalent to it, i.e., that has the same utility function. We show that for important classes of preference models, the set of possibly strictly optimal alternatives is the unique minimal equivalent subset. Secondly, we consider how to compare A to another set of alternatives B , where A and B correspond to different initial decision choices. This is closely related to the problem of computing setwise max regret. We derive mathematical results that allow different computational techniques for these problems, using linear programming, and especially, with a novel approach using the extreme points of the epigraph of the utility function.

Valuing quality in mental healthcare: A discrete choice experiment eliciting preferences from mental healthcare service users, mental healthcare professionals and the general population

High and sustained healthcare quality is important worldwide, though health policy may prioritise the achievement of certain aspects of quality over others. This study determines the relative importance of different aspects of mental healthcare quality to different stakeholders by eliciting preferences in a UK sample using a discrete choice experiment (DCE). DCE attributes were generated using triangulation between policy documents and mental healthcare service user and mental healthcare professional views, whilst ensuring attributes were measurable using available data. Ten attributes were selected: waiting times; ease of access; person-centred care; co-ordinated approach; continuity; communication, capacity and resources; treated with dignity and respect; recovery focus; inappropriate discharge; quality of life (QoL). The DCE was conducted online (December 2018 to February 2019) with mental healthcare service users (n = 331), mental healthcare professionals (n = 510), and members of the general population (n = 1018). Respondents' choices were analysed using conditional logistic regression. Relative preferences for each attribute were generated using the marginal rate of substitution (MRS) with QoL as numeraire. Across all stakeholders, being treated with dignity and respect was of high importance. A coordinated approach was important across all stakeholders, whereas communication had higher relative importance for healthcare professionals and service users and ease of access had higher relative importance for the general population. This implies that policy could be affected by the choice of whose preferences (service users, healthcare professionals or general population) to use, since this impacts on the relative value and implied ranking of different aspects of mental healthcare quality.

The use of composite time trade-off and discrete choice experiment methods for the valuation of the Short Warwick-Edinburgh Mental Well-being Scale (SWEMWBS): a think-aloud study

Purpose

To identify patterns and problems in completing composite time trade-off (C-TTO) and discrete choice experiment (DCE) exercises for the valuation of the Short Warwick-Edinburgh Mental Well-being Scale (SWEMWBS) to inform the optimisation of a valuation protocol.

Methods

Fourteen cognitive interviews were conducted in the UK using concurrent and retrospective think-aloud and probing techniques. Each participant completed 8 C-TTO tasks and 8 DCE tasks within a computer-assisted personal interview setting. Verbal information was transcribed verbatim. Axial coding and thematic analysis were used to organise the qualitative data and identify patterns and problems with the completion of tasks.

Results

While participants found the tasks generally manageable, five broad themes emerged to explain and optimise the response to the tasks. (1) Format and structure: attention to the design of practice examples, instructions, and layout were needed. (2) Items and levels: underlying relationships were discovered across different combinations of levels of SWEMWBS items. (3) Decision heuristics: participants engaged in diverse strategies to assist trade-off decisions. (4) Valuation feasibility: certain states were difficult to imagine, compare and quantify. (5) Valuation outcome: the data quality was affected by participants’ discriminatory ability across states and their time trade-off decisions.

Conclusion

The interviews contributed insights regarding the robustness of the proposed methods. The application of C-TTO and DCE valuation techniques was practical and suitable for capturing individual attitudes towards different mental well-being scenarios. A modified protocol informed by the results is being tested in a larger sample across the UK.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11136-022-03123-0.

A multi-objective supplier selection framework based on user-preferences

This paper introduces an interactive framework to guide decision-makers in a multi-criteria supplier selection process. State-of-the-art multi-criteria methods for supplier selection elicit the decision-maker's preferences among the criteria by processing pre-collected data from different stakeholders. We propose a different approach where the preferences are elicited through an active learning loop. At each step, the framework optimally solves a combinatorial problem multiple times with different weights assigned to the objectives. Afterwards, a pair of solutions among those computed is selected using a particular query selection strategy, and the decision-maker expresses a preference between them. These two steps are repeated until a specific stopping criterion is satisfied. We also introduce two novel fast query selection strategies, and we compare them with a myopically optimal query selection strategy. Computational experiments on a large set of randomly generated instances are used to examine the performance of our query selection strategies, showing a better computation time and similar performance in terms of the number of queries taken to achieve convergence. Our experimental results also show the usability of the framework for real-world problems with respect to the execution time and the number of loops needed to achieve convergence.

Transforming challenges into opportunities: conducting health preference research during the COVID-19 pandemic and beyond

The disruptions to health research during the COVID-19 pandemic are being recognized globally, and there is a growing need for understanding the pandemic’s impact on the health and health preferences of patients, caregivers, and the general public. Ongoing and planned health preference research (HPR) has been affected due to problems associated with recruitment, data collection, and data interpretation. While there are no “one size fits all” solutions, this commentary summarizes the key challenges in HPR within the context of the pandemic and offers pragmatic solutions and directions for future research. We recommend recruitment of a diverse, typically under-represented population in HPR using online, quota-based crowdsourcing platforms, and community partnerships. We foresee emerging evidence on remote, and telephone-based HPR modes of administration, with further studies on the shifts in preferences related to health and healthcare services as a result of the pandemic. We believe that the recalibration of HPR, due to what one would hope is an impermanent change, will permanently change how we conduct HPR in the future.

Trust me; I know what I am doing investigating the effect of choice list elicitation and domain-relevant training on preference reversals in decision making for others

One core assumption of standard economic theory is that an individual's preferences are stable, irrespective of the method used to elicit them. This assumption may be violated if preference reversals are observed when comparing different methods to elicit people's preferences. People may then prefer A over B using one method while preferring B over A using another. Such preference reversals pose a significant problem for theoretical and applied research. We used a sample of medical and economics students to investigate preference reversals in the health and financial domain when choosing patients/clients. We explored whether preference reversals are associated with domain-relevant training and tested whether using guided 'choice list' elicitation reduces reversals. Our findings suggest that preference reversals were more likely to occur for medical students, within the health domain, and for open-ended valuation questions. Familiarity with a domain reduced the likelihood of preference reversals in that domain. Although preference reversals occur less frequently within specialist domains, they remain a significant theoretical and practical problem. The use of clearer valuation procedures offers a promising approach to reduce preference reversals.

Public values and plurality in health priority setting: What to do when people disagree and why we should care about reasons as well as choices

CONTEXT

'What does 'The Public' think?' is a question often posed by researchers and policy makers, and public values are regularly invoked to justify policy decisions. Over time there has been a participatory turn in the social and health sciences, including health technology assessment and priority setting in health, towards citizen participation such that public policies reflect public values. It is one thing to agree that public values are important, however, and another to agree on how public values should be elicited, deliberated upon and integrated into decision-making. Surveys of public values rarely deliver unanimity, and preference heterogeneity, or plurality, is to be expected.

METHODS

This paper examines the role of public values in health policy and how to elicit, analyse, and present values, in the face of plurality. We delineate the strengths and weaknesses of aggregative and deliberative methods before setting out a new empirical framework, drawing on Sunstein's Incompletely Theorised Agreements, based on three levels: principles, policies and patients. The framework is illustrated using a recognised policy dilemma - the provision of high cost, limited-effect medicines intended to extend life for people with terminal illnesses.

FINDINGS

Application of the multi-level framework to public values permits transparent consideration of plurality, including analysis of coherence and consensus, in a way that offers routes to policy recommendations that are based on public values and justified in those terms.

CONCLUSIONS

Using the new framework and eliciting quantitative and qualitative data across levels of abstraction has the potential to inform policy recommendations grounded in public values, where values are plural. This is not to suggest that one solution will magically emerge, but rather that choices between policies can be explicitly justified in relation to the properties of public values, and a much clearer understanding of (in)consistencies and areas of consensus.

Comparison of online and face-to-face valuation of the EQ-5D-5L using composite time trade-off

OBJECTIVE

The aim of this study was to compare online, unsupervised and face-to-face (F2F), supervised valuation of EQ-5D-5L health states using composite time trade-off (cTTO) tasks.

METHODS

The official EuroQol experimental design and valuation protocol for the EQ-5D-5L of 86 health states were implemented in interviewer-assisted, F2F and unsupervised, online studies. Validity of preferences was assessed using prevalence of inconsistent valuations and expected patterns of TTO values. Respondent task engagement was measured using number of trade-offs and time per task. Trading patterns such as better-than-dead only was compared between modes. Value sets were generated using linear regression with a random intercept (RILR). Value set characteristics such as range of scale and dimension ranking were evaluated between modes.

RESULTS

Five hundred one online and 1,134 F2F respondents completed the surveys. Mean elicited TTO values were higher online than F2F when compared by health state severity. Compared to F2F, a larger proportion of online respondents did not assign the poorest EQ-5D-5L health state (i.e., 55555) the lowest TTO value ([Online] 41.3% [F2F] 12.2%) (p < 0.001). A higher percentage of online cTTO tasks were completed in 3 trade-offs or fewer ([Online] 15.8% [F2F] 3.7%), (p < 0.001). When modeled using the RILR, the F2F range of scale was larger than online ([Online] 0.600 [F2F] 1.307) and the respective dimension rankings differed.

CONCLUSIONS

Compared to F2F data, TTO tasks conducted online had more inconsistencies and decreased engagement, which contributed to compromised data quality. This study illustrates the challenges of conducting online valuation studies using the TTO approach.

Appraising patient preference methods for decision-making in the medical product lifecycle: an empirical comparison

Background

Incorporating patient preference (PP) information into decision-making has become increasingly important to many stakeholders. However, there is little guidance on which patient preference assessment methods, including preference exploration (qualitative) and elicitation (quantitative) methods, are most suitable for decision-making at different stages in the medical product lifecycle (MPLC). This study aimed to use an empirical approach to assess which attributes of PP assessment methods are most important, and to identify which methods are most suitable, for decision-makers’ needs during different stages in the MPLC.

Methods

A four-step cumulative approach was taken: 1) Identify important criteria to appraise methods through a Q-methodology exercise, 2) Determine numerical weights to ascertain the relative importance of each criterion through an analytical hierarchy process, 3) Assess the performance of 33 PP methods by applying these weights, consulting international health preference research experts and review of literature, and 4) Compare and rank the methods within taxonomy groups reflecting their similar techniques to identify the most promising methods.

Results

The Q-methodology exercise was completed by 54 stakeholders with PP study experience, and the analytical hierarchy process was completed by 85 stakeholders with PP study experience. Additionally, 17 health preference research experts were consulted to assess the performance of the PP methods. Thirteen promising preference exploration and elicitation methods were identified as likely to meet decision-makers’ needs. Additionally, eight other methods that decision-makers might consider were identified, although they appeared appropriate only for some stages of the MPLC.

Conclusions

This transparent, weighted approach to the comparison of methods supports decision-makers and researchers in selecting PP methods most appropriate for a given application.

What matters to me - a web-based preference elicitation tool for clients in long-term care: a user-centred design

Background

During the process of decision-making for long-term care, clients are often dependent on informal support and available information about quality ratings of care services. However, clients do not take ratings into account when considering preferred care, and need assistance to understand their preferences. A tool to elicit preferences for long-term care could be beneficial. Therefore, the aim of this qualitative descriptive study is to understand the user requirements and develop a web-based preference elicitation tool for clients in need of long-term care.

Methods

We applied a user-centred design in which end-users influence the development of the tool. The included end-users were clients, relatives, and healthcare professionals. Data collection took place between November 2017 and March 2018 by means of meetings with the development team consisting of four users, walkthrough interviews with 21 individual users, video-audio recordings, field notes, and observations during the use of the tool. Data were collected during three phases of iteration: Look and feel, Navigation, and Content. A deductive and inductive content analysis approach was used for data analysis.

Results

The layout was considered accessible and easy during the Look and feel phase, and users asked for neutral images. Users found navigation easy, and expressed the need for concise and shorter text blocks. Users reached consensus about the categories of preferences, wished to adjust the content with propositions about well-being, and discussed linguistic difficulties.

Conclusion

By incorporating the requirements of end-users, the user-centred design proved to be useful in progressing from the prototype to the finalized tool ‘What matters to me’. This tool may assist the elicitation of client’s preferences in their search for long-term care.

Do time trade-off values fully capture attitudes that are relevant to health-related choices?

Previous research has shown that demographics, beliefs, and self-reported own health influence TTO values. Our hypothesis is that attitudes towards length and quality of life influence TTO values, but should no longer affect a set of related choices that are based on respondents' own TTO scores. A representative sample of 1339 respondents was asked their level of agreement to four statements relating to the importance of quality and length of life. Respondents then went on to value 4 EQ-5D 5L states using an online interactive survey and a related set of 6 pairwise health-related choice questions, set up, so that respondents should be indifferent between choice options. We explored the impact of attitudes using regression analysis for TTO values and a logit model for choices. TTO values were correlated with the attitudes and were found to have a residual impact on the choices. In particular, those respondents who preferred quality of life over length of life gave less weight to the differences in years and more weight to differences in quality of life in these choice. We conclude that although the TTO responses reflect attitudes, these attitudes continue to affect health-related choices.

Understanding the adoption and use of point-of-care tests in Dutch general practices using multi-criteria decision analysis

BACKGROUND

The increasing number of available point-of-care (POC) tests challenges clinicians regarding decisions on which tests to use, how to efficiently use them, and how to interpret the results. Although POC tests may offer benefits in terms of low turn-around-time, improved patient's satisfaction, and health outcomes, only few are actually used in clinical practice. Therefore, this study aims to identify which criteria are, in general, important in the decision to implement a POC test, and to determine their weight. Two POC tests available for use in Dutch general practices (i.e. the C-reactive protein (CRP) test and the glycated haemoglobin (HbA_1c) test) serve as case studies. The information obtained from this study can be used to guide POC test development and their introduction in clinical practice.

METHODS

Relevant criteria were identified based on a literature review and semi-structured interviews with twelve experts in the field. Subsequently, the criteria were clustered in four groups (i.e. user, organization, clinical value, and socio-political context) and the relative importance of each criterion was determined by calculating geometric means as implemented in the Analytic Hierarchy Process. Of these twelve experts, ten participated in a facilitated group session, in which their priorities regarding both POC tests (compared to central laboratory testing) were elicited.

RESULTS

Of 20 criteria in four clusters, the test's clinical utility, its technical performance, and risks (associated with the treatment decision based on the test result) were considered most important for using a POC test, with relative weights of 22.2, 12.6 and 8.5%, respectively. Overall, the experts preferred the POC CRP test over its laboratory equivalent, whereas they did not prefer the POC HbA_1c test. This difference was mainly explained by their strong preference for the POC CRP test with regard to the subcriterion 'clinical utility'.

CONCLUSIONS

The list of identified criteria, and the insights in their relative impact on successful implementation of POC tests, may facilitate implementation and use of existing POC tests in clinical practice. In addition, having experts score new POC tests on these criteria, provides developers with specific recommendations on how to increase the probability of successful implementation and use.

Quantified patient preferences for lifestyle intervention programs for diabetes prevention-a protocol for a systematic review

BACKGROUND

The 20-70% participation of diabetes patients in lifestyle interventions (LSI) worldwide seems to be rather sub-optimal, in spite of all intents of such interventions to delay further progress of the disease. Positive effects through LSI are expected in particular for patients who suffer less from diabetes-related limitations or other chronic diseases. Seeing that diabetes prevalence and with it mortality are increasing, LSI have become an inherent part of diabetes treatment standards. Various qualitative studies have been carried out to identify participation barriers for LSI. However, these have not resulted in more detailed knowledge about the relative importance of factors with an inhibiting impact on participation. Since it cannot be assumed that all of the influencing factors have equivalent values, it is necessary to investigate their individual importance with regard to a positive or negative decision about participating. There are no systematic reviews on patient preferences for LSI programs in diabetes prevention. As a result, the main objectives of this systematic review are to (i) identify existing patient preference elicitation studies related to LSI for diabetic patients, (ii) summarize the methods applied and findings, and (iii) appraise the reporting and methodological quality of such studies.

METHODS

We will perform systematic literature searches to identify suitable studies from 14 electronic databases. Retrieved study records will be included based on predefined eligibility criteria as defined in this protocol. We will run abstract and full-text screenings and then extract data from all selected studies by filling in a predefined data extraction spreadsheet. We will undertake a descriptive, narrative synthesis of findings to address the study objectives, since no pooling for quantified preferences is for methodological reasons implementable. We will pay special attention to aspects of methodological quality of preference elicitation by applying established evaluation criteria of the ISPOR and some own developed criteria for different elicitation techniques. All critical stages within the screening, data extraction, and synthesis processes will be conducted by two pairs of authors. This protocol adheres to PRISMA and PRISMA-P standards.

DISCUSSION

The proposed systematic review will provide an overview of the methods used and current practice in the elicitation and quantification of patients' preferences for diabetes prevention lifestyle interventions. Furthermore, the methodological quality of the identified studies will be appraised as well.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO CRD42018086988.

Priority setting in health: development and application of a multi-criteria algorithm for the population of New Zealand's Waikato region

Background

Priority setting in a climate of diverse needs and limited resources is one of the most significant challenges faced by health care policymakers. This paper develops and applies a comprehensive multi-criteria algorithm to help determine the relative importance of health conditions that affect a defined population.

Methods

Our algorithm is implemented in the context of the Waikato District Health Board (WDHB) in New Zealand, which serves approximately 10% of the New Zealand population. Strategic priorities of the WDHB are operationalized into five criteria along which the algorithm is structured—scale of disease, household financial impact of disease, health equity, cost-effectiveness, and multimorbidity burden. Using national-level data and published literature from New Zealand, the World Health Organization, and other high-income Commonwealth countries, 25 health conditions in Waikato are identified and mapped to these five criteria. These disease-criteria mappings are weighted with data from an ordered choice survey administered to the general public of the Waikato region. The resulting output of health conditions ranked in order of relative importance is validated against an explicit list of health concerns, provided by the survey respondents.

Results

Heart disease and cancerous disorders are assigned highest priority rankings according to both the algorithm and the survey data, suggesting that our model is aligned with the primary health concerns of the general public. All five criteria are weighted near-equal across survey respondents, though the average health equity preference score is 9.2% higher for Māori compared to non-Māori respondents. Older respondents (50 years and above) ranked issues of multimorbidity 4.2% higher than younger respondents.

Conclusions

Health preferences of the general population can be elicited using ordered-choice surveys and can be used to weight data for health conditions across multiple criteria, providing policymakers with a practical tool to inform which health conditions deserve the most attention. Our model connects public health strategic priorities, the health impacts and financial costs of particular health conditions, and the underlying preferences of the general public. We illustrate a practical approach to quantifying the foundational criteria that drive public preferences, for the purpose of relevant, legitimate, and evidence-based priority setting in health.

Electronic supplementary material

The online version of this article (10.1186/s12962-018-0121-z) contains supplementary material, which is available to authorized users.

Resource allocation in NHS dentistry: recognition of societal preferences (RAINDROP): study protocol

Background

Resources in any healthcare systems are scarce relative to need and therefore choices need to be made which often involve difficult decisions about the best allocation of these resources. One pragmatic and robust tool to aid resource allocation is Programme Budgeting and Marginal Analysis (PBMA), but there is mixed evidence on its uptake and effectiveness. Furthermore, there is also no evidence on the incorporation of the preferences of a large and representative sample of the general public into such a process. The study therefore aims to undertake, evaluate and refine a PBMA process within the exemplar of NHS dentistry in England whilst also using an established methodology (Willingness to Pay (WTP)) to systematically gather views from a representative sample of the public.

Methods

Stakeholders including service buyers (commissioners), dentists, dental public health representatives and patient representatives will be recruited to participate in a PBMA process involving defining current spend, agreeing criteria to judge services/interventions, defining areas for investment and disinvestment, rating these areas against the criteria and making final recommendations. The process will be refined based on participatory action research principles and evaluated through semi-structured interviews, focus groups and observation of the process by the research team. In parallel a representative sample of English adults will be recruited to complete a series of four surveys including WTP valuations of programmes being considered by the PBMA panel. In addition a methodological experiment comparing two ways of eliciting WTP will be undertaken.

Discussion

The project will allow the PBMA process and particularly the use of WTP within it to be investigated and developed. There will be challenges around engagement with the task by the panel undertaking it and with the outputs by stakeholders but careful relationship building will help to mitigate this. The large volume of data will be managed through careful segmenting of the analysis and the use of the well-established Framework approach to qualitative data analysis. WTP has various potential biases but the elicitation will be carefully designed to minimise these and some methodological investigation will take place.

Electronic supplementary material

The online version of this article (10.1186/s12913-018-3302-8) contains supplementary material, which is available to authorized users.

A scenario-based MCDA framework for wastewater infrastructure planning under uncertainty

Wastewater infrastructure management is increasingly important because of urbanization, environmental pollutants, aging infrastructures, and climate change. We propose a scenario-based multi-criteria decision analysis (MCDA) framework to compare different infrastructure alternatives in terms of their sustainability. These range from the current centralized system to semi- and fully decentralized options. Various sources of uncertainty are considered, including external socio-economic uncertainty captured by future scenarios, uncertainty in predicting outcomes of alternatives, and incomplete preferences of stakeholders. Stochastic Multi-criteria Acceptability Analysis (SMAA) with Monte Carlo simulation is performed, and rank acceptability indices help identify robust alternatives. We propose step-wise local sensitivity analysis, which is useful for practitioners to effectively elicit preferences and identify major sources of uncertainty. The approach is demonstrated in a Swiss case study where ten stakeholders are involved throughout. Their preferences are quantitatively elicited by combining an online questionnaire with face-to-face interviews. The trade-off questions reveal a high concern about environmental and an unexpectedly low importance of economic criteria. This results in a surprisingly good ranking of high-tech decentralized wastewater alternatives using urine source separation for most stakeholders in all scenarios. Combining scenario planning and MCDA proves useful, as the performance of wastewater infrastructure systems is indeed sensitive to socio-economic boundary conditions and the other sources of uncertainty. The proposed sensitivity analysis suggests that a simplified elicitation procedure is sufficient in many cases. Elicitation of more information such as detailed marginal value functions should only follow if the sensitivity analysis finds this necessary. Moreover, the uncertainty of rankings can be considerably reduced by better predictions of the outcomes of alternatives. Although the results are case based, the proposed decision framework is generalizable to other decision contexts.

The impact of a belief in life after death on health-state preferences: True difference or artifact?

PURPOSE

In most religions, the preservation of one's own, God-given, life is considered obligatory, while the time trade-off method (TTO) forces one to voluntarily forego life years. We sought to verify how this conflict impacts TTO-results among the religious.

METHODS

We used the data from the only EQ-5D valuation in Poland (2008, three-level, 321 respondents, 23 states each)-a very religious, mostly Catholic country. We measured the religiosity with the belief in afterlife question on two levels: strong (definitely yes) and some (also rather yes), both about a third of the sample.

RESULTS

The religious more often are non-traders, unwilling to give up any time in exchange for quality of life: odds ratio (OR) equal to 1.97 (strong religiosity), OR 1.55 (some religiosity); and less often consider a state worse than death: OR 0.67 (strong), OR 0.81 (some). These associations are statistically significant ([Formula: see text]) and hold when controlling for possible demographic confounders. Strong religiosity abates the utility loss: in the additive approach by 0.14, in the multiplicative approach by the factor of 2.1 (both [Formula: see text]), especially among the older. Removing the effect of religiosity from the value set reduces the utility by 0.05 on average.

CONCLUSION

The results may stem from a true difference in preferences or be a TTO-artifact and would vanish for other elicitation methods. Juxtaposing our findings with comments from respondents in other studies suggests the latter. Therefore, this Weltanschauung effect should be removed in cost-utility analysis.

Paying for Forest Ecosystem Services: Voluntary Versus Mandatory Payments

The emergence of new markets for forest ecosystem services can be a compelling opportunity for market diversification for private forest landowners, while increasing the provision of public goods from private lands. However, there is limited information available on the willingness-to-pay (WTP) for specific forest ecosystem services, particularly across different ecosystem market mechanisms. We utilize survey data from Oregon and Washington households to compare marginal WTP for forest ecosystem services and the total WTP for cost-effective bundles of forest ecosystem services obtained from a typical Pacific Northwest forest across two value elicitation formats representing two different ecosystem market mechanisms: an incentive-compatible choice experiment involving mandatory tax payments and a hypothetical private provision scenario modeled as eliciting contributions to the preferred forest management alternative via a provision point mechanism with a refund. A representative household's total WTP for the average forest management program was estimated at $217.59 per household/year under a mandatory tax mechanism and $160.44 per household/per year under a voluntary, crowdfunding-style, contribution mechanism; however, these estimates are not statistically different. Marginal WTP estimates were assessed for particular forest ecosystem service attributes including water quality, carbon storage, mature forest habitat, and public recreational access. This study finds that survey respondents place significant economic value on forest ecosystem services in both elicitation formats and that the distributions of the marginal WTP are not statistically significantly different.

Preference elicitation approach for measuring the willingness to pay for liver cancer treatment in Korea

Background/Aims

The Korean government has expanded the coverage of the national insurance scheme for four major diseases: cancers, cardiovascular diseases, cerebrovascular diseases, and rare diseases. This policy may have a detrimental effect on the budget of the national health insurance agency. Like taxes, national insurance premiums are levied on the basis of the income or wealth of the insured.

Methods

Using a preference elicitation method, we attempted to estimate how much people are willing to pay for insurance premiums that would expand their coverage for liver cancer treatment.

Results

We calculated the marginal willingness to pay (MWTP) through the marginal rate of substitution between the two attributes of the insurance premium and the total annual treatment cost by adopting conditional logit and mixed logit models.

Conclusions

The effects of various other terms that could interact with socioeconomic status were also estimated, such as gender, income level, educational attainment, age, employment status, and marital status. The estimated MWTP values of the monthly insurance premium for liver cancer treatment range from 4,130 KRW to 9,090 KRW.

Set-valued dynamic treatment regimes for competing outcomes

Dynamic treatment regimes (DTRs) operationalize the clinical decision process as a sequence of functions, one for each clinical decision, where each function maps up-to-date patient information to a single recommended treatment. Current methods for estimating optimal DTRs, for example Q-learning, require the specification of a single outcome by which the "goodness" of competing dynamic treatment regimes is measured. However, this is an over-simplification of the goal of clinical decision making, which aims to balance several potentially competing outcomes, for example, symptom relief and side-effect burden. When there are competing outcomes and patients do not know or cannot communicate their preferences, formation of a single composite outcome that correctly balances the competing outcomes is not possible. This problem also occurs when patient preferences evolve over time. We propose a method for constructing DTRs that accommodates competing outcomes by recommending sets of treatments at each decision point. Formally, we construct a sequence of set-valued functions that take as input up-to-date patient information and give as output a recommended subset of the possible treatments. For a given patient history, the recommended set of treatments contains all treatments that produce non-inferior outcome vectors. Constructing these set-valued functions requires solving a non-trivial enumeration problem. We offer an exact enumeration algorithm by recasting the problem as a linear mixed integer program. The proposed methods are illustrated using data from the CATIE schizophrenia study.

The use of alternative preference elicitation methods in complex discrete choice experiments

We analyse stated preference data over nursing jobs collected from two different discrete choice experiments: a multi-profile case best-worst scaling experiment (BWS) prompting selection of the best and worst among alternative jobs, and a profile case BWS wherein the respondents choose the best and worst job attributes. The latter allows identification of additional utility parameters and is believed to be cognitively easier. Results suggest that respondents place greater value on pecuniary over non-pecuniary gains in the multi-profile case. There is little evidence that this discrepancy is induced by the extra cognitive burden of processing several profiles at once in the multi-profile case. We offer thoughts on other likely mechanisms.