Can we fix it? Yes we can! But what? A new test of procedural invariance in TTO-measurement

The effect of duration and time preference on the gap between adult and child health state valuations in time trade-off

Composite time trade-off (cTTO) utilities have been found to be higher when adults value health states for children than for themselves. It is not clear if these differences reflect adults assigning truly higher utilities to the same health state in different perspectives, or if they are caused by other factors, which are not accounted for in the valuation procedure. We test if the difference between children's and adults' cTTO valuations changes if a longer duration than the standard 10 years is used. Personal interviews with a representative sample of 151 adults in the UK were conducted. We employed the cTTO method to estimate utilities of four different health states, where adults considered states both from their own and a 10-year-old child's perspective, for durations of 10 and 20 years. We corrected the cTTO valuations for perspective-specific time preferences in a separate task, again for both perspectives. We replicate the finding that cTTO utilities are higher for the child perspective than for the adult perspective, although the difference is only significant when controlling for other variables in a mixed effects regression. Time preferences are close to 0 on average, and smaller for children than adults. After correcting TTO utilities for time preferences, the effect of perspective is no longer significant. No differences were found for cTTO tasks completed with a 10- or 20-year duration. Our results suggest that the child-adult gap is partially related to differences in time preferences and, hence, that correcting cTTO utilities for these preferences could be useful.

Multivariate risk preferences in the quality-adjusted life year model

The interest in multivariate and higher-order risk preferences has increased. A growing body of literature has demonstrated the relevance and impact of these preferences, but for health the evidence is lacking. We measure multivariate and higher-order risk preferences for quality of life (QoL) and longevity, the two attributes of the Quality-Adjusted Life Year (QALY) model. We observe preferences for a positive correlation between these attributes and for pooling together a fixed loss in one of the attributes and a mean-zero risk in the other, and for pooling together mean-zero risks in QoL and longevity. The findings indicate that higher-order risk preferences are stronger for health than for money. Furthermore, we test if preferences for a risky treatment for a disease affecting only QoL, depend on life expectancy. We find no such a relation, but there is a positive relation between riskiness of a comorbidity affecting life expectancy and risk aversion for a QoL treatment. We therefore observe no definitive deviation from the QALY model, although the model is more robust when expected longevity is high. Our findings suggest that the current practice of cost-effectiveness analysis should be generalized to account for risk aversion in QoL and longevity, and higher-order preferences.

Trust me; I know what I am doing investigating the effect of choice list elicitation and domain-relevant training on preference reversals in decision making for others

One core assumption of standard economic theory is that an individual's preferences are stable, irrespective of the method used to elicit them. This assumption may be violated if preference reversals are observed when comparing different methods to elicit people's preferences. People may then prefer A over B using one method while preferring B over A using another. Such preference reversals pose a significant problem for theoretical and applied research. We used a sample of medical and economics students to investigate preference reversals in the health and financial domain when choosing patients/clients. We explored whether preference reversals are associated with domain-relevant training and tested whether using guided 'choice list' elicitation reduces reversals. Our findings suggest that preference reversals were more likely to occur for medical students, within the health domain, and for open-ended valuation questions. Familiarity with a domain reduced the likelihood of preference reversals in that domain. Although preference reversals occur less frequently within specialist domains, they remain a significant theoretical and practical problem. The use of clearer valuation procedures offers a promising approach to reduce preference reversals.

Discounting in Economic Evaluations

Appropriate discounting rules in economic evaluations have received considerable attention in the literature and in national guidelines for economic evaluations. Rightfully so, as discounting can be quite influential on the outcomes of economic evaluations. The most prominent controversies regarding discounting involve the basis for and height of the discount rate, whether costs and effects should be discounted at the same rate, and whether discount rates should decline or stay constant over time. Moreover, the choice for discount rules depends on the decision context one adopts as the most relevant. In this article, we review these issues and debates, and describe and discuss the current discounting recommendations of the countries publishing their national guidelines. We finish the article by proposing a research agenda.

Are people consistent when trading time for health?

The conventional, or standard, time trade-off (TTO) procedure, which is used to elicit the values that people place on health states that are in turn required to calculate quality adjusted life-years (QALYs), asks respondents to trade off fewer life years for better health. It is possible to reverse the procedure to ask respondents to trade off less health for more life years. Theoretically, these two procedures should generate the same TTO values for any given health state. This article reports that for health states defined by differing frequencies of migraine attack, the standard TTO gives health state values that are significantly higher than those given by the reverse TTO. The observed systematic procedural invariance, which substantiates some previous findings reported in the literature and is consistent with a loss aversion effect, challenges the validity of the TTO for generating reliable valuations of health states.

Deriving time discounting correction factors for TTO tariffs

The Time Trade-off (TTO) method is a popular method for valuing health state utilities and is frequently used in economic evaluations. However, this method produces utilities that are distorted by several biases. One important bias entails the failure to incorporate time discounting. This paper aims to measure time discounting for health outcomes in a sample representative for the general population. In particular, we estimate TTO scores alongside time discounting in order to derive a set of correction factors that can be employed to correct raw TTO scores for the downward bias caused by time discounting. We find substantial positive correction factors, which are increasing with the severity of the health state. Furthermore, higher discounting is found when using more severe health states in the discounting elicitation task. More research is needed to further develop discount rate elicitation procedures and test their validity, especially in general public samples. Moreover, future research should investigate the correction of TTO values for other biases as well, such as loss aversion, and to develop a criterion to test the external validity of TTO scores.

Time trade-off: one methodology, different methods

There is no scientific consensus on the optimal specification of the time trade-off (TTO) task. As a consequence, studies using TTO to value health states may share the core element of trading length of life for quality of life, but can differ considerably on many other elements. While this pluriformity in specifications advances the understanding of TTO from a methodological point of view, it also results in incomparable health state values. Health state values are applied in health technology assessments, and in that context comparability of information is desired. In this article, we discuss several alternative specifications of TTO presented in the literature. The defining elements of these specifications are identified as being either methodological, procedural or analytical in nature. Where possible, it is indicated how these elements affect health state values (i.e., upward or downward). Finally, a checklist for TTO studies is presented, which incorporates a list of choices to be made by researchers who wish to perform a TTO task. Such a checklist enables other researchers to align methodologies in order to enhance the comparability of health state values.

Converting condition-specific measures into preference-based outcomes for use in economic evaluation

Quality-adjusted life-years (QALYs) play an important role in reimbursement decisions when one of the criteria is the cost-effectiveness of the health technology. While for many generic QALYs (e.g., based on the EQ-5D) are viewed as the gold standard, there has been a considerable increase in interest in using condition-specific data to generate QALYs. There are two main methods: mapping from the condition-specific data to a generic health-related quality of life measure; and direct valuation of condition-specific health states. Whether one believes condition-specific data are useful even if generic QALY data are available, or simply that condition-specific data are helpful in the absence of generic measures of health-related quality of life, it is timely to review recent research activity directed at making greater use of condition-specific data to inform assessments of cost-effectiveness.

The way that you do it? An elaborate test of procedural invariance of TTO, using a choice-based design

The time tradeoff (TTO) method is often used to derive Quality-Adjusted Life Year health state valuations. An important problem with this method is that results have been found to be responsive to the procedure used to elicit preferences. In particular, fixing the duration in the health state to be valued and inferring the duration in full health that renders an individual indifferent, causes valuations to be higher than when the duration in full health is fixed and the duration in the health state to be valued is elicited. This paper presents a new test of procedural invariance for a broad range of time horizons, while using a choice-based design and adjusting for discounting. As one of the known problems with the conventional procedure is the violation of constant proportional tradeoffs (CPTO), we also investigate CPTO for the alternative TTO procedure. Our findings concerning procedural invariance are rather supportive for the TTO procedure. We find no violations of procedural invariance except for the shortest gauge duration. The results for CPTO are more troublesome: TTO scores depend on gauge duration, reinforcing the evidence reported when using the conventional procedure.

A direct method for measuring discounting and QALYs more easily and reliably

Time discounting and quality of life are two important factors in evaluations of medical interventions. The measurement of these two factors is complicated because they interact. Existing methods either simply assume one factor given, based on heuristic assumptions, or invoke complicating extraneous factors, such as risk, that generate extra biases. The authors introduce a method for measuring discounting (and then quality of life) that involves no extraneous factors and that avoids distorting interactions. Their method is considerably simpler and more realistic for subjects than existing methods. It is entirely choice based and thus can be founded on economic rationality requirements. An experiment demonstrates the feasibility of this method and its advantages over classical methods.

Ordering errors, objections and invariance in utility survey responses: a framework for understanding who, why and what to do

BACKGROUND

Utilities are the quantification of the perceived quality of life associated with any health state. They are used to calculate QALYs, the outcome measure in cost-utility analysis. Generally measured through surveys of individuals, utilities often contain apparent or unapparent errors that can bias resulting values and QALYs calculated from these values.

OBJECTIVE

The aim of this study was to improve direct health utility elicitation methodology through the identification of the types of survey responses that indicate errors and objections, and the reasons underlying them.

METHODS

We conducted a systematic review of the medical (PubMed), economics (EconLit) and psychology (PsycINFO) literature from 1975 through June 2010 for articles describing the types and frequency of errors and objections in directly elicited utility survey responses, and strategies to address these responses. Primary data were collected through an internet-based utility survey (standard gamble) of community members to identify responses that indicate error or objections. A qualitative telephone survey was conducted among a subset of respondents with these types of responses using an open-ended protocol to elicit rationales for them.

RESULTS

A total of 11 papers specifically devoted to errors, objections and invariance in utility responses have been published since the mid-1990s. Error/objection responses can be broadly categorized into ordering errors (which include illogical and inconsistent responses) and objections/invariance (which include missing data, protest responses and refusals to trade time or risk in utility questions). Reported frequencies of respondents making ordering errors ranged from 5% to 100%, and up to 35% of respondents have been reported as objecting to the survey or task in some manner. Changes in the design, administration and analysis of surveys can address these potentially problematic responses. Survey data (n = 398) showed that individuals who provided invariant responses (n = 26) reported the lowest level of difficulty with the survey and often identified as religious (23% of invariant responders found the survey difficult vs 63% of all responders, and 77% of invariant responders identified as religious compared with 56% of entire sample; p < 0.05 for both). Respondents who provided illogical responses (n = 50) were less likely to be college educated (56% of illogical responders vs 73% of entire sample; p < 0.05), and less likely to be confident in their responses (62% vs 75% of entire sample; p < 0.05). Qualitative interviews (n = 42) following the survey revealed that the majority of ordering errors were a result of confusion, lack of attention or difficulty in responding to the survey on the part of the respondent, while invariant responses were often considered and thoughtful reactions to the premise of valuing health using the standard gamble task.

CONCLUSIONS

Rationales for error/objection responses include difficulty in articulating preferences or misunderstanding with a complex survey task, and also thoughtful and considered protestations to the task. Mechanisms to correct unintentional errors may be useful, but cannot address intentional responses to elements of the measurement task. Identification and analysis of the prevalence of errors and objections in responses in utility data sets are essential to understanding the accuracy and precision of utility estimates and analyses that depend thereon.

On the (not so) constant proportional trade-off in TTO

Purpose

The linear and power QALY models require that people in Time Trade-off (TTO) exercises sacrifice the same proportion of lifetime to obtain a health improvement, irrespective of the absolute amount. However, evidence on these constant proportional trade-offs (CPTOs) is mixed, indicating that these versions of the QALY model do not represent preferences. Still, it may be the case that a more general version of the QALY model represents preferences. This version has the property that people want to sacrifice the same proportion of utilities of lifetime for a health improvement, irrespective of the amount of this lifetime.

Methods

We use a new method to correct TTO scores for utility of life duration and test whether decision makers trade off utility of duration and quality at the same rate irrespective of duration.

Results

We find a robust violation of CPTO for both uncorrected and corrected TTO scores. Remarkably, we find higher values for longer durations, contrary to most previous studies. This represents the only study correcting for utility of life duration to find such a violation.

Conclusions

It seems that the trade-off of life years is indeed not so constantly proportional and, therefore, that health state valuations depend on durations.

The influence of subjective life expectancy on health state valuations using a 10 year TTO

AIM

To investigate if subjective life expectancy (SLE) impacts the willingness to trade-off (WTT) and the number of years traded-off in a 10-years time trade-off (TTO) exercise to obtain health state valuations.

METHODS

An Internet-based questionnaire was administered in a sample representative for the Dutch general public. Next to basic demographic characteristics and SLE, respondents were asked to perform three TTO exercises. The following EQ-5D health states were included 21211 (TTO1), 22221 (TTO2) and 33312 (TTO3). The WTT was studied using a probit regression model. The number of years traded-off was investigated using a generalized negative binomial regression model. The independent variables used in both models were age, gender, quality of life, education, the difference between age and expected age of death (SLE), and a variable indicating whether the SLE was less than 10 years (SLE<10).

RESULTS

Three hundred and thirty nine respondents completed the questionnaire. The mean utility scores were 0.96 (TTO1), 0.94 (TTO2) and 0.79 (TTO3). The probit model showed that SLE was the only variable with a significant influence on WTT. The gnbreg showed that the number of years traded-off was also significantly influenced by SLE. In addition, age and education significantly influenced the number of years traded-off.

CONCLUSION

The WTT years and the number of years traded-off were both influenced by SLE in 10-years TTO exercises. Reducing remaining life expectancy to 10 years in a TTO may thus increase loss aversion and, especially in respondents losing relatively many expected life years, diminish WTT and the amount of time traded off.

The correction of TTO-scores for utility curvature using a risk-free utility elicitation method

This paper describes and employs a new method to correct time tradeoff (TTO)-scores for utility of life duration curvature. In contrast to most previous attempts to do so, it uses a risk-free method that corresponds well to the risk-free properties of the TTO-method. In addition, the method is robust to several biases that occur under methods that incorporate risk. Our results show a significant degree of curvature in utility of life duration and therefore a clear bias in TTO-scores. The risk-free method seems to be useful to correct TTO-scores for this influence and leads to significantly higher quality-adjustment factors.