A program of methodological research to arrive at the new international EQ-5D-5L valuation protocol

OBJECTIVES

To describe the research that has been undertaken by the EuroQol Group to improve current methods for health state valuation, to summarize the results of an extensive international pilot program, and to outline the key elements of the five-level EuroQol five-dimensional (EQ-5D-5L) questionnaire valuation protocol, which is the culmination of that work.

METHODS

To improve on methods of health state valuation for the EQ-5D-5L questionnaire, we investigated the performance of different variants of time trade-off and discrete choice tasks in a multinational setting. We also investigated the effect of three modes of administration on health state valuation: group interviews, online self-completion, and face-to-face interviews.

RESULTS

The research program provided the basis for the EQ-5D-5L questionnaire valuation protocol. Two different types of tasks are included to derive preferences: a newly developed composite time trade-off task and a forced-choice paired comparisons discrete choice task. Furthermore, standardized blocked designs for the selection of the states to be valued by participants were created and implemented together with all other elements of the valuation protocol in a digital aid, the EuroQol Valuation Technology, which was developed in conjunction with the protocol.

CONCLUSIONS

The EuroQol Group has developed a standard protocol, with accompanying digital aid and interviewer training materials, that can be used to create value sets for the EQ-5D-5L questionnaire. The use of a well-described, consistent protocol across all countries enhances the comparability of value sets between countries, and allows the exploration of the influence of cultural and other factors on health state values.

Assessing the performance of the EQ-VAS in the NHS PROMs programme

Objectives

The study aims to increase knowledge about the performance of the EuroQol-visual analogue scales (EQ-VAS) in the UK NHS patient-reported outcome measures (PROMs) programme, which covers groin hernia, hip and knee replacement and varicose vein surgery, and make suggestions for improved collection, coding and analysis of data.

Methods

Four hundred scanned images of matched before-and-after EQ-VAS PROMs responses were selected at random. These were classified according to the different ways in which they were completed. Patient-level PROMs programme data linked to Hospital Episode Statistics for all patients from April 2009 to February 2011 were used to analyse the relationship between the EQ-VAS and the EQ-5D profile, index-weighted profile and condition-specific instruments. The linked PROMs and HES data comprise 331,951 anonymised patient records.

Results

A large majority (95 %) of EQ-VAS responses were completed in an unambiguous way, but only a minority (45 %) conformed strictly to the instructions given, posing challenges for data coding. The EQ-VAS data have a predictable and consistent relationship with the EQ-5D profile, although the correlations between the EQ-VAS and other measures of patient-reported health, both before and after surgery and in the change between them, are weak.

Conclusions

EQ-VAS data might be improved by providing better guidance on collection and coding. It is argued that the observed differences in results from EQ-VAS and other measures of health reflect the fact that it measures a broader underlying construct of health, arguably providing a means of summarising overall health that is closer to the patient’s perspective.

Truly inefficient or providing better quality of care? Analysing the relationship between risk-adjusted hospital costs and patients' health outcomes

Observed variation in hospital costs may be attributable to differences in patients' health outcomes. Previous studies have resorted to inherently incomplete outcome measures such as mortality or re-admission rates to assess this claim. This study makes use of a novel dataset of routinely collected patient-reported outcome measures (PROMs) linked to inpatient records to (i) access the degree to which cost variation is associated with variation in patients' health gain and (ii) explore how far judgement about hospital cost performance changes when health outcomes are accounted for. We use multilevel modelling to address the clustering of patients in providers and isolate unexplained cost variation. We find some evidence of a U-shaped relationship between risk-adjusted costs and outcomes for hip replacement surgery. For three other procedures (knee replacement, varicose vein and groin hernia surgery), the estimated relationship is sensitive to the choice of PROM instrument. We do not observe substantial changes in cost performance estimates when outcomes are explicitly accounted for.

Lead versus lag-time trade-off variants: does it make any difference?

OBJECTIVES

The traditional time trade-off (TTO) method has some problems in the valuation of health states considered worse than dead. The aim of our study is to compare two TTO variants that address this issue: lead-time and lag-time TTO.

METHODS

Quota sampling was undertaken in June 2011 in Buenos Aires as part of the EQ-5D-5L Multinational Pilot Study. Respondents were randomly assigned to one of the TTO variants with two blocks of five EQ-5D-5L health states. Tasks were administered using a web-based digital aid (EQ-VT) administered in a group interview.

RESULTS

A total of 387 participants were included [mean age 38.85 (SD: 13.97); 53.14 % females]. The mean observed values ranged from 0.44 (0.59) for state 21111 to 0.02 (0.76) for state 53555 in the lead-time group and between 0.53 (0.52) and 0.08 (0.76) in the lag-time group. There were no statistically significant differences in the values between TTO variants, except for a significant difference of 0.19 for state 33133. In both variants, marked peaks were observed around the value 0 across all states, with a higher percentage of 0 responses in the last state valued, suggesting ordering effects.

CONCLUSIONS

No important differences were found between TTO variants regarding values for EQ-5D-5L health states, suggesting that they could be equivalent variants. However, differences between the two methods may have been obscured by other aspects of the study design affecting the characteristics of the data.

Time to tweak the TTO: results from a comparison of alternative specifications of the TTO

This article examines the effect that different specifications of the time trade-off (TTO) valuation task may have on values for EQ-5D-5L health states. The new variants of the TTO, namely lead-time TTO and lag-time TTO, along with the classic approach to TTO were compared using two durations for the health states (15 and 20 years). The study tested whether these methods yield comparable health-state values. TTO tasks were administered online. It was found that lag-time TTO produced lower values than lead-time TTO and that the difference was larger in the longer time frame. Classic TTO values most resembled those of the lag-time TTO in a 20-year time frame in terms of mean absolute difference. The relative importance of different domains of health was systematically affected by the duration of the health state. In the tasks with a 10-year health-state duration, anxiety/depression had the largest negative impact on health-state values; in the tasks with a 5-year duration, the pain/discomfort domain had the largest negative impact.

One-to-one versus group setting for conducting computer-assisted TTO studies: findings from pilot studies in England and the Netherlands

We compare two settings for administering time trade-off (TTO) tasks in computer-assisted interviews (one-to-one, interviewer-led versus group, self-complete) by examining the quality of the data generated in pilot studies undertaken in England and the Netherlands. The two studies used near-identical methods, except that in England, data were collected in one-to-one interviews with substantial amounts of interviewer assistance, whereas in the Netherlands, the computer aid was used as a self-completion tool in group interviews with lesser amounts of interviewer assistance. In total, 801 members of the general public (403 in England; 398 in the Netherlands) each completed five TTO valuations of EQ-5D-5L health states. Respondents in the Netherlands study showed a greater tendency to give 'round number' values such as 0 and 1 and to complete tasks using a minimal number of iterative steps. They also showed a greater tendency to skip the animated instructions that preceded the first task and to take into account assumptions that they were specifically asked not to take into account. When faced with a pair of health states in which one state dominated the other, respondents in the Netherlands study were more likely than those in the England study to give a higher value to the dominant health state. On the basis of these comparisons, we conclude that the one-to-one, interviewer-led setting is superior to the group, self-complete setting in terms of the quality of data generated and that the former is more suitable than the latter for TTO studies being used to value EQ-5D-5L.

The effects of lead time and visual aids in TTO valuation: a study of the EQ-VT framework

BACKGROUND

The effect of lead time in time trade-off (TTO) valuation is not well understood. The purpose of this study was to investigate the effects on health-state valuation of the length of lead time and the way the lead-time TTO task is displayed visually.

METHODS

Using two general population samples, we compared three lead-time TTO variants: 10 years of lead time in full health preceding 5 years of unhealthy time (standard); 5 years of lead time preceding 5 years of unhealthy time (experimental); and 10 years of lead time and 5 years of unhealthy time, presented with a visual aid to highlight the point where the lead time ends (experimental). Participants were randomized to receive one of the lead-time variants, as administered by a computer software program.

RESULTS

Health-state values generated by TTO valuation tasks using a longer lead time were slightly lower than those generated by tasks using a shorter lead time. When lead time and unhealthy time were presented with visual aids highlighting the difference between the lead time and unhealthy time, respondents spent more time considering health states with a value close to 0.

CONCLUSIONS

Different lead-time time trade-off variants should be carefully studied in order to achieve the best measurement of health-state values using this new method.

Lead versus lag-time trade-off variants: does it make any difference?

OBJECTIVES

The traditional time trade-off (TTO) method has some problems in the valuation of health states considered worse than dead. The aim of our study is to compare two TTO variants that address this issue: lead-time and lag-time TTO.

METHODS

Quota sampling was undertaken in June 2011 in Buenos Aires as part of the EQ-5D-5L Multinational Pilot Study. Respondents were randomly assigned to one of the TTO variants with two blocks of five EQ-5D-5L health states. Tasks were administered using a web-based digital aid (EQ-VT) administered in a group interview.

RESULTS

A total of 387 participants were included [mean age 38.85 (SD: 13.97); 53.14 % females]. The mean observed values ranged from 0.44 (0.59) for state 21111 to 0.02 (0.76) for state 53555 in the lead-time group and between 0.53 (0.52) and 0.08 (0.76) in the lag-time group. There were no statistically significant differences in the values between TTO variants, except for a significant difference of 0.19 for state 33133. In both variants, marked peaks were observed around the value 0 across all states, with a higher percentage of 0 responses in the last state valued, suggesting ordering effects.

CONCLUSIONS

No important differences were found between TTO variants regarding values for EQ-5D-5L health states, suggesting that they could be equivalent variants. However, differences between the two methods may have been obscured by other aspects of the study design affecting the characteristics of the data.

A uniform time trade off method for states better and worse than dead: feasibility study of the 'lead time' approach

The way time trade off (TTO) values are elicited for states of health considered 'worse than being dead' has important implications for the mean values used in economic evaluation. Conventional approaches to TTO, as used in the UK's 'MVH' value set, are problematic because they require fundamentally different trade-offs tasks for the valuation of states better and worse than dead. This study aims to refine and test the feasibility of a new approach described by Robinson and Spencer (2006. Health Economics 15: 393-402), and to explore the characteristics of the valuation data it generates. The approach introduces a 'lead time' into the TTO, producing a uniform procedure for generating values either >0 or<0. We used this lead time TTO to value 10 moderate to severe EQ-5D states using a sample of the general public (n=109). We conclude that the approach is feasible for use in valuation studies and appears to overcome the discontinuity in values around 0 evident in conventional methods. However, further research is required to resolve the issue of how to handle participants who 'use up' all lead time; to develop ways of controlling for individual time preferences; and to better understand the implications for valuations of states better than dead.

Patient-reported outcome measures in the NHS: new methods for analysing and reporting EQ-5D data

In a landmark move, the UK Department of Health (DH) has introduced the routine collection of patient-reported outcome measures (PROMs) to measure the performance of health-care providers. From April 2009, generic (EQ-5D) and condition-specific PROMs are being collected from patients before and after four surgical procedures; eventually this will be extended to include a wide range of other NHS services. The aim of this article is to report analysis of the EQ-5D data generated from a pilot study commissioned by the DH and to consider the implications for the use of EQ-5D data in performance indicators and measures of patient benefit. We present two new methods that we have developed for analysing and displaying EQ-5D profile data: a Paretian Classification of Health Change and a health profile grid. We show that EQ-5D profile data can be readily analysed to generate insights into the nature of changes in patient-reported health that would be obscured by summarising these profiles by their index scores, or focusing just on the post operative outcomes. Our methods indicate differences between providers and between sub-groups of patients. Our results also show striking differences in changes in EQ-5D profiles between surgical procedures, which require further investigation.

An analysis of NICE's 'restricted' (or 'optimized') decisions

BACKGROUND

A common way of describing UK National Institute for Health and Clinical Excellence (NICE) decisions is to distinguish between cases where NICE recommended use of a healthcare technology by all relevant patients ('yes'); those where it did not recommend use ('no'); and those where its decisions are a mixture of 'yes' to some patient subgroups, and 'no' to others. Over half of NICE's decisions are of this mixed type, which involve restricting (or 'optimizing') patient use in some way.

OBJECTIVE

To report an attempt to develop a robust and defensible means of measuring and describing the degree of patient access in mixed NICE decisions.

METHODS

A list of mixed decisions made from 2006 to the end of 2009 was identified using HTAinSite™. The following calculation was used: M = (p/P) × 100, where M is a measure of the level of patient access (0 = no access, 100 = full access), P is the set of patients considered in the guidance as Potential candidates for treatment (given the licensed use and the scope of NICE's appraisal), and p is a subset of those patients, for whom NICE did recommend treatment. M can be estimated either for a specific product or for a group of technologies (Multiple Technology Appraisals). Both product-specific and overall M were estimated, using estimates of p obtained from NICE costing templates. These data are subject to some important limitations, so the results should be regarded as illustrative.

RESULTS

Of the 69 medicines that have received a mixed decision since January 2006, 34 included details that allowed the estimation of M. Of these 34 decisions, 24 (71%) had a product-specific M ≤50, 16 (47%) M ≤25 and 11 (32%) M ≤10. That is, in just under three-quarters of the mixed decisions for which P and p were available, NICE recommended use for less than half of patients for whom the medicine is licensed, and in nearly one-third of these sorts of decisions, NICE recommended use in ≤10% of potential patients. The estimates of M for groups of technologies provide a slightly different picture: for example, grouped M was ≤10 in <20% of decisions.

CONCLUSIONS

The measure of patient access, M, proposed here has the potential to provide a more informative way of reporting all NICE decisions, particularly 'restricted' (or 'optimized') decisions.

"Yes", "No" or "Yes, but"? Multinomial modelling of NICE decision-making

The National Institute for Health and Clinical Excellence (NICE) issues mandatory guidance on health technologies to the UK NHS, based on clinical evidence, cost-effectiveness and other considerations. However, the exact factors considered, their relative importance and tradeoffs between them are not made explicit. Previous research modelled NICE decisions as a binary choice (accept/reject) dependent on cost-effectiveness, amongst other variables. This paper proposes and tests an alternative model of decision-making that may better represent the "yes, but..." nature of many NICE decisions. Decisions were categorised as "recommended for routine use", "recommended for restricted use" or "not recommended". The NICE appraisal process was modelled as a single decision between the three categories. Multinomial logistic regression techniques were used to evaluate the impact of: quantity/quality of clinical evidence; cost-effectiveness; decision date; existence of alternative treatments; budget impact; technology type. Results suggest that interventions supported by more randomised trials are more likely to be recommended and endorsed for routine use. Higher cost-effectiveness ratios increased the likelihood of interventions being rejected rather than recommended for restricted use but did not significantly affect the decision between routine and restricted use. Pharmaceuticals, interventions appraised early in the NICE programme and those with more systematic reviews were also less likely to be rejected, while patient group submissions made a recommendation for routine rather than restricted use more likely. The presence of factors affecting the decision between routine and restricted use but not that between routine use and rejection suggests that modelling these three outcomes reflects NICE decision-making more closely than binary-choice analyses.

Understanding health state valuations: a qualitative analysis of respondents' comments

Self-completed questionnaires (usually distributed by post) and visual analogue scales (VAS) are common means for collecting valuations for hypothetical EQ-5D health states. Although opportunities for respondents to comment on aspects of the exercise are often included, these data have rarely been the focus of analyses. This paper, therefore, reports on solicited and unsolicited written comments received in a New Zealand survey in 1999 to which 1360 people responded of whom approximately 50% made comments. The comments were systematically analysed via an inductive process that allowed principal themes to emerge with respect to understanding the peculiarities of respondents' valuations, particularly common 'data problems', and their perceptions as to the adequacy or otherwise of the EQ-5D classification system. From our findings we conclude that the valuation exercise imposes a substantial cognitive burden on respondents and many do not understand it (for a variety of reasons documented in the paper). Also, although there is some evidence for expanding the EQ-5D (generic) health-related quality of life dimensions, more research as to whether this holds for a larger sample and across cultural settings is needed. We offer recommendations for future research into understanding respondents' cognitive processes and possible revisions to the design and administration of the EQ-5D.

The validity and reliability of EQ-5D health state valuations in a survey of Măori

This note reports on a 2000 study of the content validity of the EQ-5D's representation of health for 66 Măori (New Zealand's indigenous people, comprising 14.5% of the population) accessed through cultural networks. Also examined was the construct validity of the health state valuation instrument and its test-retest reliability based on repeated valuations for the two extreme health states. The possibility that the EQ-SD fails to capture what Măori regard as 'health' derives from the so-called 'Măori health model' that augments biological health with mental, spiritual and family well-being. Seventy six percent of respondents considered the EQ-5D's representation of health to be adequate. This proportion is not statistically significantly different from the rates for non-Măori and Măori respectively in an earlier study and might suggest the EQ-5D has content validity for Măori. However, the high prevalence of missing valuations, particularly for dead, and logical inconsistencies suggests that the health state valuation instrument lacks construct validity, although there is evidence of test retest reliability.

Logical inconsistencies in survey respondents' health state valuations -- a methodological challenge for estimating social tariffs

Logical inconsistencies in survey respondents' valuations of hypothetical health states - represented by the EQ-5D, for example - present a conundrum as to whether or not their responses ought to be included for estimating social 'tariffs'. A 'logical inconsistency' occurs when a state that 'in logical terms' is unambiguously less severe than another is assigned a lower value. Excluding such responses is defensible on data quality grounds but puts at risk the representativeness of the estimated tariff, given it is meant to represent the preferences of 'society'. This paper explores the rationale for and effect of excluding, to varying degrees, responses distinguished by the number of pairwise inconsistencies they contain, and reports equations for two tariffs that arise from contrasting approaches. The data are from a random sample of adult New Zealanders whose visual analogue scale valuations for a selection of EQ-5D states were collected in 1999 via a postal survey to which 1360 people responded (a 50% response rate). We conclude that there is no simple, generalisable 'rule' to guide exclusions and therefore researchers ought to explore the sensitivity of their estimated tariffs (and ultimately QALY estimates) to alternative treatments of logically inconsistent responses.

The social costs of alcohol abuse in New Zealand

AIMS

This study updates and extends previous New Zealand research on the social costs of alcohol abuse.

DESIGN

This economic cost study used the human capital approach.

SETTING

New Zealand, 1991.

PARTICIPANTS

The total New Zealand population.

MEASUREMENTS

The estimated cost of alcohol abuse for 1 year included direct and indirect costs. Costs such as lost production resulting from premature death and sickness, reduced working efficiency and excess unemployment comprised indirect costs. Direct costs included hospital costs, accident compensation payments, police and justice system costs. A range of social cost estimates was constructed based on various prevalence rates of alcohol abuse, discount rates for lost production and the excess unemployment rate.

FINDINGS

Using a range of assumptions regarding the proportion of each event attributable to alcohol, the sum of social costs ranged from $1045 million to $4005 million in 1991. The direct costs ranged from $341 million to $589 million, respectively.

CONCLUSIONS

While providing an indication of the societal impact of alcohol abuse, these costs pertain to a relatively narrow range of alcohol-related effects. The paper identifies a number of areas where further research is required.

The cost-effectiveness of mammography screening: evidence from a microsimulation model for New Zealand

Mammography screening currently represents the only means by which the mortality rate from breast cancer can be modified substantially. A national mammography screening programme is being considered for New Zealand, and pilot programmes were established in two regions (Otago/Southland and Waikato) in 1991 to determine the potential costs and benefits of mammography for New Zealand women. The aim of this paper is to explore the cost-effectiveness of mammography screening in New Zealand relative to no screening, and to examine the marginal change in costs and benefits of altering programme characteristics such as the age of women invited and screening frequency. Cost-effectiveness is measured by the net cost (the costs of screening minus the treatment savings averted by the early detection of cancers) per year of life gained, from the perspective of the public health care sector. A microsimulation computer model, MICROLIFE, was developed to facilitate the estimation of mortality reduction and cost-effectiveness. The results show that, while mammography screening does not 'save money' overall, the cost per year of life saved for a range of policies compares favourably with other New Zealand health services, and is comparable to the results from economic evaluations of mammography screening overseas. Of those regimes considered, screening women 50-64 years of age at 3-yearly intervals appears to be most cost-effective.

The costs of mammography screening in New Zealand: evidence from the pilot programmes

AIMS

To measure the public health service costs associated with New Zealand's pilot mammography screening programmes. To compare the early evidence on cost per woman screened and per cancer detected in those programmes to that of overseas screening programmes. To estimate the cost of introducing a national screening programme in New Zealand.

METHODS

Costs in each screening centre were obtained by a careful examination of screening budgets and public health service accounts; these were inflation adjusted using a consumers price index, and analysed in terms of equivalent annual operating costs.

RESULTS

In the first year of screening the cost per woman screened (in $1991) was $182 in Waikato and $178 in Otago/Southland. The cost per woman screened in the third year of screening (with an assumed full screening throughput of 8,000 women per annum) is estimated to fall to $106 and $113 for the Waikato and Otago/Southland programmes respectively. The cost per cancer detected in the first screening round differs between the two programmes. In the first year of screening the cost per cancer detected was $35,975 in Waikato and $21,908 in Otago/Southland. The difference was primarily attributable to a lower cancer detection rate in Waikato in that period (0.51% of women screened compared with 0.81% in Otago/Southland).

CONCLUSIONS

The initial performance of the New Zealand pilot programmes, both in terms of cost per woman screened and cost per cancer detected, falls within the range indicated from overseas experience. An established national screening programme is estimated to add between $9.3 and $9.9 million dollars (in 1991 dollar terms) to health service costs each year. These costs will be partly offset by savings resulting from the earlier detection of cancers.

The effects of denturism: New Zealand dentists' response to competition

The New Zealand Dental Act of 1988 allowed clinical dental technicians to deal directly with the public in fitting and supplying dentures. This study tested the hypothesis that dentists responded to competition from dental technicians by lowering their fees. The results indicate that there was no significant change in the fees charged by dentists for dentures. The apparent failure of deregulation to produce the expected outcome could be due to the competitive pressure imposed by dental technicians practicing illegally prior to 1988, to consumers' lack of information, or to barriers to "consumer search" imposed by the act itself.

The economics of dental practice in New Zealand, 1974-1993

New Zealand Dental Association survey data were used to explore trends in the principal indicators of dental practice economics over the last two decades. Gross income in the most recent financial year, 1992, was the highest during this period. However, the costs of dental practice have also risen, and now absorb close to 60 percent of revenue compared with around 50 percent in the early 1970s. As a result, real net income has yet to return to the "peak" levels achieved in the mid-1970s. Nevertheless, average real net incomes earned in 1992 (in 1988 dollar terms) of $85,701 is around 8 percent higher than the average real net income between 1974 and 1992 of $79,243. Furthermore, the lower income tax rates applying in the post-1990 period have produced post-1990 after-tax incomes which are higher in real terms than those in the pre-1990 period. The trends in real net dental income earned by dentists between 1982 and 1992 are shown to correspond closely to the trends in the average incomes of higher-income consumers. The rate of increase in the fees charged by dentists between 1978 and 1993 has, for most services, exceeded the rate of increase in consumer prices generally. A notable exception is the fee charged for complete dentures, which displayed much more modest increases. It is possible that this is attributable to the presence of competing suppliers (dental technicians) for this service.(ABSTRACT TRUNCATED AT 250 WORDS)

The distribution of household expenditures on health care

AIMS

To find out whether New Zealand household expenditures on health care services vary according to the income of the household. To compare expenditures on health care in 1987 with 1991.

METHODS

Information about household income and expenditure on health services was obtained from the Department of Statistics annual household expenditure and income surveys for the 1987 and 1991 financial years. Four categories of health expenditure were examined: general practitioner fees, dental fees, optician and optometrist fees, and spending on all health services combined.

RESULTS

Spending on health care is unequally distributed across income groups. In particular, the highest income households spend six times as much on dental care as the lowest income households. The difference between high and low income households in the amount spent on all health services was greater in 1991 than in 1987. In 1991 high income households spent 3.6 times as much on health services as low income households, compared with three times as much in 1987.

CONCLUSIONS

High income households spend substantially more on health care than do low income households. Households appear to assign a higher priority to medical care than dental care, although this may reflect the lack of any state subsidy on adult dental care.