Measuring patient preferences: rating scale versus standard gamble

Developing a risk-based composite neurologic outcome for a trial of hydroxyurea in young children with sickle cell disease

BACKGROUND

Studies of interventions to prevent the many neurological complications of sickle cell disease must take into account multiple outcomes of variable severity, with limited sample size. The goals of the studies presented were to use investigator preferences across outcomes to determine an attitude-based weighting of relevant clinical outcomes and to establish a valid composite outcome for a clinical trial.

METHODS

In Study 1, investigators were surveyed about their practice regarding hydroxyurea therapy and opinions about outcomes for the "Hydroxyurea to Prevent the Central Nervous System Complications of Sickle Cell Disease Trial" (HU Prevent), and their minimally acceptable relative risk reduction for the two outcome components, motor and neurocognitive deficits. In Study 2, HU Prevent investigators provided overall weights for these two components. In Study 3, they provided more granular rankings, ratings, and maximum number acceptable to harm. A weighted composite outcome, the Stroke Consequences Risk Score, was constructed that incorporates the major neurologic complications of sickle cell disease. The Stroke Consequences Risk Score represents the 3-year risk of suffering the adverse consequences of stroke. In Study 4, the results of the Optimizing Primary Stroke Prevention in Sickle Cell Anemia (STOP2) and Silent Infarct Transfusion Trials were reanalyzed in light of the composite outcome.

RESULTS

In total, 22 to 27 investigators participated per study. In Study 1, across three samplings between 2009 and 2015, the average minimally acceptable relative risk reduction ranged from 0.36 to 0.50, at or below the target effect size of 0.50. In 2015, 21 (91%) reported that a placebo-controlled trial is reasonable; 23 (100%), that it is ethical; and 22 (96%), that they would change their practice, if the results of the trial were positive. In Studies 2 and 3, the weight elicited for a cognitive decline (of 10 IQ points) from the overall assessment was 0.67 (and for motor deficit, the complementary 0.33); from ranking, 0.6; from rating, 0.58; and from maximal number acceptable to harm, 0.5. Using data from two major clinical trials, Study 4 demonstrated the same conclusions as the original trials using the Stroke Consequences Risk Score, with smaller p-values for both reanalyses. An assessment of acceptability was performed as well.

CONCLUSION

This set of studies provides the rationale, justification, and validation for the use of a weighted composite outcome and confirms the need for the phase III HU Prevent study. Surveys of investigators in multi-center studies can provide the basis of clinically meaningful outcomes that foster the translation of study results into practice while increasing the efficiency of a study.

Desiderata for sharable computable biomedical knowledge for learning health systems

In this commentary, we work out the specific desired functions required for sharing knowledge objects (based on statistical models) presumably to be used for clinical decision support derived from a learning health system, and, in so doing, discuss the implications for novel knowledge architectures. We will demonstrate how decision models, implemented as influence diagrams, satisfy the desiderata. The desiderata include locally validate discrimination, locally validate calibration, locally recalculate thresholds by incorporating local preferences, provide explanation, enable monitoring, enable debiasing, account for generalizability, account for semantic uncertainty, shall be findable, and others as necessary and proper. We demonstrate how formal decision models, especially when implemented as influence diagrams based on Bayesian networks, support both the knowledge artifact itself (the "primary decision") and the "meta-decision" of whether to deploy the knowledge artifact. We close with a research and development agenda to put this framework into place.

Feasibility, reliability, and validity of three health-state valuation methods using multiple-outcome vignettes on moderate-risk pregnancy at term

OBJECTIVES

Preference-based health-state valuation methods such as discrete choice experiment (DCE) are claimed to be superior than attitude-based valuation methods like visual analogue scale (VAS) and time trade-off (TTO). We compared VAS, TTO, and DCE in terms of feasibility, reliability, and validity using vignettes depicting moderate-risk pregnancy at term.

METHODS

People from the community (n = 97) participated in both a panel session and an individual home assignment. Each participant valuated 46 vignettes with VAS, TTO, and DCE. Each vignette consisted of five attributes: maternal health antepartum, time between diagnosis and delivery, process of delivery, maternal outcome, and neonatal outcome. The questionnaire included Feasibility, which we evaluated by questionnaire. Test–retest reliability and interobserver consistency were assessed by intraclass correlation (ICC), and variance consistency by generalization theory. Convergent validity was determined with ICC and Cohen's kappa; construct validity was determined with linear regression, multinomial logit modeling, and Kendall's Tau-b correlation (τ).

RESULTS

The DCE was reported as most feasible (DCE: 87% vs. VAS: 69% vs. TTO: 42%). Test–retest reliability was high overall and equal (VAS: ICC = 0.77; TTO: ICC = 0.79; DCE: κ = 0.78). The VAS had the highest interobserver reliability (ICC = 0.73). Convergent validity between VAS and DCE was high (κ = 0.79) and there was sufficient construct validity between VAS and DCE (τ = 0.68). The TTO yielded less optimal results. Generally, neonatal and maternal outcomes weighed most, whereas process outcomes weighed least in moderate-risk pregnancy at term.

CONCLUSIONS

In our context of multidimensional health states with complex trade-offs, DCE was superior to TTO and performed equal to VAS, with DCE displaying slightly higher user feasibility.

Health utilities in mild cognitive impairment and dementia: a population study in Sweden

OBJECTIVE

To collect new primary data on community-based health utilities (time trade-off values) in different stages of mild cognitive impairment and dementia from a general population sample.

METHODS

A cross-sectional study including 1,800 randomly selected members of the Swedish public aged 45-84 was performed through a postal survey; 42% response rate. The Clinical Dementia Rating scale was used for describing progressive stages of dementia in vignettes that were used in combination with time trade-off questions for valuing the perceived quality of life in these stages.

RESULTS

The time-trade off values varied considerably across the progressive disease stages. The mean score was 0.82 for mild cognitive impairment, 0.62 for mild dementia, 0.40 for moderate dementia, and 0.25 for severe dementia. In multiple regression analyses, the scores were relatively insensitive to demographic factors like age, gender and self-assessed health.

CONCLUSIONS

The results showed that the average time trade-off values declined sharply from mild cognitive impairment to progressing stages of dementia. Since there are many methodological challenges involved in measuring health state utilities in mild cognitive impairment and dementia, more research that evaluates different methods would be welcome.

Overcoming inherent problems of preference-based techniques for measuring health benefits: an empirical study in the context of kidney transplantation

Background

Economic valuations of health care programs often require using patients as subjects, implying that research methodology should conform to the surrounding social, cultural and ethical context. The significance of patients' opinions in health care decisions has been well defined but in Greece, and perhaps elsewhere, clinicians remain skeptical. The purpose of this study was to investigate, for the first time in Greece, the feasibility of measuring preference-based health-state utilities and willingness to pay and to determine the context-based adaptations required to overcome inherent elicitation problems.

Methods

A survey including a time trade-off (TTO), a standard gamble (SG), and two willingness-to-pay (WTP) questions was self-administered to a homogenous group of 606 end stage renal disease patients in 24 dialysis facilities throughout Greece and the overall response rate was 78.5%. Typical elicitation methods were adapted to overcome methodological problems such as subjective life expectancy and question framing. Spearman's correlation coefficients were calculated between utilities and WTP and parametric tests (independent samples t-test and ANOVA) examined score differences as a result of demographic and clinical factors.

Results

Mean health-state utilities were 72.56 (TTO) and 91.06 (SG) and these were statistically significantly different (P < 0.0005). Significant correlations, in the expected directions, were observed between TTO – SG, TTO – WTP and SG – WTP (P < 0.01). High ceiling effects were observed in the TTO and SG methods indicating patients' adversity to risk and unwillingness to trade-off life years. Higher WTP was observed from younger patients (P < 0.0005), males (P < 0.05), higher education levels (P < 0.01), single (P < 0.0005) and employed (P < 0.005).

Conclusion

This study demonstrated, to a fair extent, that adapting research methods to context-based particularities does not necessarily compromise results and should be considered in situations where standard methods cannot be applied. On the other hand, it is emphasized that the results from this study are preliminary and should be interpreted cautiously until further research demonstrates the practicality, reliability and validity of alternative measurement approaches.

The relationship between health perception and utility in heart failure patients in a clinical trial: results from an OVERTURE substudy

BACKGROUND

Cost-effectiveness analyses should be based on incremental years of life gained adjusted with a health status measure known as a utility. Measuring utilities for all subjects in a large-scale randomized trial, however, would be prohibitively cumbersome. We therefore sought to estimate utilities for all subjects from results obtained in a subset of patients.

METHODS AND RESULTS

We studied a subset of patients enrolled in a randomized trial of omapatrilat for the treatment of heart failure. Survey instruments (a time trade-off questionnaire, a visual analog scale [VAS] score of overall health perception, and the Duke Activity Status Index [DASI]) were administered to patients by mail and by telephone interviews. There was a significant (P <.0001) relationship between VAS score and utility described by the power function u=1-(1-v)q, where q=2.17 (95% CI 1.76 to 2.58). There was a significant positive correlation (r=.17, P <.04) between the DASI and utility, and a significant negative correlation (r=-.26, P <.001) between utility and New York Heart Association functional class.

CONCLUSION

There is a significant relationship between the relatively easily obtainable health perception score by VAS with the more complex utility by time tradeoff for a subset of patients in a multicenter randomized clinical trial. This relationship may be helpful in examining the cost-effectiveness of new treatments for heart failure.

Cost-utility analysis in evaluating prophylaxis in haemophilia

Prophylaxis is an expensive form of management in haemophilia but has demonstrated many advantages with respect to decreasing joint bleeds and potentially preventing joint damage. The valuation of prophylaxis and how the costs and benefits of this intervention compare with other interventions in the management of haemophiliacs can be evaluated through cost-utility analysis (CUA). CUA is an economic method of analysis where the benefits of a healthcare intervention are expressed as an overall utility or preference, usually in the form of quality-adjusted life years (QALYs). This is a composite measure, which takes into consideration both an individual's lifespan and quality of life (QoL). The most difficult aspect of performing a CUA is the measurement of health-related QoL (HRQoL). Much work is ongoing into evaluating HRQoL in haemophiliacs. This paper addresses some of the ways in which this can be achieved and some of the problems with evaluating HRQoL. Ultimately CUA may provide a tool to allow societies to decide if prophylaxis is worth the cost and how the costs and benefits of prophylaxis compare to other healthcare interventions for other disease entities.

Validity of a modified standard gamble elicited from parents of a hospital-based cohort of children

OBJECTIVES

To examine the validity of a modified standard gamble (Mod SG) (nondeath baseline) by comparing these scores to SG (death baseline), time trade off (TTO), visual analog scale (VAS), Health Utilities Index (HUI), and Child Health Questionnaire (CHQ).

METHODS

Respondents were parents of in-patients with cancer receiving chemotherapy and parents of children without cancer attending outpatient clinics. Construct validity was determined by comparing a priori hypotheses to actual correlations between measures. Discriminant validity was examined by anticipating that in-patients with cancer would have lower HRQL than outpatients.

RESULTS

85 families were included. Both Mod SG and SG were moderately correlated with TTO (r=0.50 and r=0.49; P<.01 for both). Both Mod SG and SG were moderately correlated with TTO (r=0.47 and r=0.05, P<0.002 for both).

CONCLUSION

The Mod SG did not perform better than SG. Two nonoverlapping groups of HRQL measures were demonstrated.

Developing, implementing, and evaluating decision support systems for shared decision making in patient care: a conceptual model and case illustration

The importance of including patient preferences in decisions regarding their care has received increased emphasis over recent years. Medical informatics can play an important role in improving patient-centered care by developing decision support systems to support the inclusion of patient preferences in clinical decision making. However, development of such systems is a complex task that requires the integration of knowledge from four major research areas: (1) the clinical domain, for understanding of the decision problem, (2) decision science and research on shared decision making, to provide the theoretical underpinnings and techniques for eliciting patient preferences; (3) medical informatics, to provide the technology and algorithms for the collection, processing, structure, presentation and integration of patient preferences into patient care; and (4) organizational knowledge, to adapt the decision support system to the practices and work flows of clinicians and the organizational and professional context of the clinical practice settings. This paper describes a conceptual model comprising eight key components that are important to be considered in the development, implementation, and evaluation of decision support systems for shared decision making in patient care. The example of CHOICE, a decision support system to assist nurses in eliciting and integrating rehabilitation patients' preferences for functional performance in patient care is used to illustrate the eight components.

Properties of the 15D and EQ-5D utility measures in a community sample of people with epilepsy

To perform economic evaluation studies in epilepsy using utilities, such as cost-utility analyses, it is necessary to have reliable and valid instruments for eliciting people's preferences. The objective of this study was to assess the reliability and validity of two multiattribute utility measures in a community sample of 397 people with epilepsy. We assessed the test-retest reliability of individual questionnaire items and aggregate scores. Additionally, construct validity was assessed by correlating items of the EQ-5D with items of the 15D health status questionnaire, and discriminant validity was assessed by comparing scores for known groups. The test-retest reliabilities for the individual items of the 15D (weighted kappa 0.59-0.90, except mobility with only 0.28) and the EQ-5D (weighted kappa 0.49-0.86) were acceptable. For the composite utility scores the test-retest reliability was better (intraclass correlation coefficient 0.93 for both 15D and the EQ-5D). Spearman's rank correlations between EQ-5D single item scores and corresponding 15D single item scores were high (rho 0.34-0.79) and generally higher than the associations between non-corresponding items. Some EQ-5D and 15D items discriminated well between patients according to seizure status, psychiatric comorbidity and working status; less well after antiepileptic drug use and neurologic comorbidity. Both the EQ-5D and 15D were reliable instruments and showed properties supporting the construct validity of both measures.

Comparing treatment valuations between and within subjects in clinical trials: does it make a difference?

Valuations may be sensitive to biases, especially if elicited alongside randomized clinical trials. We investigated the construction of valuations assigned by women who entered a randomized clinical trial and were allocated to in-hospital or domiciliary monitoring. Women assigned valuations (0-10 visual analogue scale) to the strategy they had been allocated to and to the alternative strategy. Valuations were expressed as a between-subject difference (assigned by the women allocated to the respective strategies) and as within-subject differences (assigned by all women). Domiciliary monitoring was valued higher by the women allocated to that strategy (P = 0.10). In-hospital monitoring was valued higher by the women allocated to that strategy (P = 0.02). The average within-subject differences differed by allocated strategy (P<0.01). The within-subject valuation differences showed large variability between and within groups. An overrepresentation of women favoring domiciliary monitoring and asymmetric treatment experience inflated the average within-subject difference in the domiciliary group but deflated that difference in the in-hospital group. Neither the average between-subject difference nor the average within-subject differences are free of bias. Other study designs probably cannot prevent bias. Comparing within-subject and between-subject differences is instructive.

Patient preferences for early discharge after laparoscopic cholecystectomy

Patients may have concerns about their ability to manage postoperative symptoms at home after ambulatory surgery. We assessed patients' attitudes toward postoperative care at home or in the hospital after laparoscopic cholecystectomy. Thirty-eight patients undergoing elective laparoscopic cholecystectomy were pre- and postoperatively (within a week each) presented with scenarios describing symptoms of differing severity in either a home or hospital setting and were asked to rank and rate the relative desirability of the scenarios using rating scale, standard gamble, and willingness-to-pay techniques. Preoperatively, 16 (42%), 21 (55%), and 30 (79%) patients ranked pain of mild, moderate, and severe levels, respectively, as worse than the respective levels of nausea and vomiting. Of 24 patients, 19 (79%) preferred home care to hospital care for mild symptoms, and 12 of 22 patients (55%) preferred home care to hospital care for moderate symptoms. The average ratings were 20, 53, and 90 for mild, moderate, and severe symptoms, respectively, where 0 = no symptoms and 100 = the worst symptoms possible. Patients who preferred care outside the home indicated that they were willing to pay a mean of $142 (maximum $410) as a maximal copayment to have postoperative care in the hospital and a mean of $255 to receive care in a medical hotel-like facility. Postoperative assessment correlated highly with the preoperative assessment (correlation coefficient >0.6 for rating, standard gamble, and willingness-to-pay assessments). We conclude that patients vary in their attitudes toward where they would like to receive postoperative care. Attitudes assessed preoperatively may predict their attitudes postoperatively.

IMPLICATIONS

Patients preferred to be at home for mild postoperative symptoms but in the hospital for worse postoperative symptoms. Preferences did not change with different methods of asking and were the same pre- and postoperatively. If patients made choices for their care before their procedure, they would still be happy with those decisions postoperatively.

Patient preferences for heart failure treatment: utilities are valid measures of health-related quality of life in heart failure

BACKGROUND

Current standards hold that cost-effectiveness analyses should incorporate measures of both quantity and quality of life, and that quality of life in this context is best measured by a utility. We sought to measure utility scores for patients with heart failure and to assess their validity as measures of health-related quality of life (HRQL).

METHODS AND RESULTS

We studied 50 patients with heart failure. We measured utilities with the time trade-off technique, exercise capacity with a 6-minute walk test, and HRQL with the Minnesota Living With Heart Failure questionnaire, the Medical Outcomes Study Short Form-36 (SF-36) questionnaire, and a visual analogue score. Validity was assessed by establishing correlation between utilities and these other measures. Mean utility score was 0.77 +/- 0.28. There were significant (P < .05) curvilinear relationships between utility score and visual analogue score, the physical function summary scale of the SF-36, 6-minute walk distance, and the Living With Heart Failure score. Utility scores on retest at 1 week were unchanged in a subset of 12 patients. Utilities did not vary systematically with age, sex, or ethnicity.

CONCLUSION

Utilities are valid measures of HRQL in patients with heart failure, and cost-effectiveness analyses of heart failure treatments incorporating utilities in the outcome measure can be meaningful.

Measuring patient preferences by willingness to pay to avoid: the case of acute otitis media

The aim of this study was to evaluate willingness to pay (WTP) to avoid as a method of eliciting relative values for use in expected-value (EV) decision making. Parents' preferences for the events and outcomes associated with acute otitis media (AOM) and its treatment were quantified by means of a questionnaire asking how much they would be willing to pay to avoid them. Their responses were then used to calculate the EVs of treating or not treating presumed AOM with antibiotics. The advantages of the WTP method were its simplicity, its analogy with everyday financial transactions, its explicit recognition of illness and its management as involving decreases in value, and its face validity. The disadvantages included the need to use another method (the standard gamble) to derive a value for death and the wide ranges and the poor test-retest reliability of individual parents' responses. Nonetheless, median WTP values and their ranges may prove useful in defining for physicians and policymakers the parameters of their practical management decisions. In the case of AOM, the EV of treating with antibiotics was, for the aggregate sample and for most individual parents, robustly superior because of parents' desire to avoid any increased risk of their children's death.

Quality of life in epilepsy: comparison of four preference measures

Several specific and general measures are available for the assessment of overall health related quality of life in epilepsy. Few of the commonly used measures provide utility weights for use in cost-utility analyses. This study compares four methods for measuring utility weights: time trade-off (TTO), standard gamble (SG), 15D, end the EuroQol visual analog scale. All patients aged 18-67 years with a diagnosis of epilepsy, who had been admitted to or attended the outpatient clinic at a large county hospital 1987-1994, received a comprehensive questionnaire. From 397 respondents, 82 patients were randomly selected. Most of the 57 patients completing the study generally had well-controlled epilepsy, but were still on anti-epileptic medication. Mean age was 44 years. Fourty-one percent were male and 59% female. The resulting utility weights differed considerably between the measures, both with regard to central tendency and dispersion. Median utility scores: EuroQol visual analog scale 0.75, 15D 0.90, TTO 0.98, SG 0.99. There was a good association between the EuroQol rating scale and the 15D, and a moderate association between SG and TTO. These preference instruments measure different aspects of health-related quality of life and thus yield different results. Caution should be taken when interpreting cost-utility studies, as results will depend on the choice of utility instrument.