Feasibility, reliability, and validity of three health-state valuation methods using multiple-outcome vignettes on moderate-risk pregnancy at term

Eliciting a value set for the Swedish Capability-Adjusted Life Years instrument (CALY-SWE)

PURPOSE

Our aim was to elicit a value set for Capability-Adjusted Life Years Sweden (CALY-SWE); a capability-grounded quality of life instrument intended for use in economic evaluations of social interventions with broad consequences beyond health.

METHODS

Building on methods commonly used in the quality-adjusted life years EQ-5D context, we collected time-trade off (TTO) and discrete choice experiment (DCE) data through an online survey from a general population sample of 1697 Swedish participants. We assessed data quality using a score based on the severity of inconsistencies. For generating the value set, we compared different model features, including hybrid modeling of DCE and TTO versus TTO data only, censoring of TTO answers, varying intercept, and accommodating for heteroskedasticity. We also assessed the models' DCE logit fidelity to measure agreement with potentially less-biased DCE data. To anchor the best capability state to 1 on the 0 to 1 scale, we included a multiplicative scaling factor.

RESULTS

We excluded 20% of the TTO answers of participants with the largest inconsistencies to improve data quality. A hybrid model with an anchor scale and censoring was chosen to generate the value set; models with heteroskedasticity considerations or individually varying intercepts did not offer substantial improvement. The lowest capability weight was 0.114. Health, social relations, and finance and housing attributes contributed the largest capability gains, followed by occupation, security, and political and civil rights.

CONCLUSION

We elicited a value set for CALY-SWE for use in economic evaluations of interventions with broad social consequences.

Test-Retest Reliability of EQ-5D-Y-3L Best-Worst Scaling Choices of Adolescents and Adults

BACKGROUND

There is an increasing interest to obtain adolescents' own health state valuation preferences and to understand how these differ from adult preferences for the same health state. An important question in health state valuation is whether adolescents can report preferences reliably, yet research remains limited.

OBJECTIVE

This study aims to investigate the test-retest reliability of best-worst scaling (BWS) to elicit adolescent preferences compared with adults.

METHODS

Identical BWS tasks designed to value 3-level version of EQ-5D-Y health states were administered online in samples of 1000 adolescents (aged 11-17 years) and 1006 adults in Spain. The valuation survey was repeated approximately 3 days later. We calculated (1) simple percentage agreement and (2) kappa statistic as measures of test-retest reliability. We also compared BWS marginal frequencies and relative attribute importance between baseline and follow-up to explore similarities in the obtained preferences.

RESULTS

We found that both adolescents and adults were able to report their preferences with moderate reliability (kappa: 0.46 for adolescents, 0.46 for adults) for best choices and fair to moderate reliability (kappa: 0.39 for adolescents, 0.41 for adults) for worst choices. No notable difference was observed across years of child age. Higher consistency was observed for best choices than worst in some dimensions for both populations. No significant differences were found in the relative attribute importance between baseline and follow-up in both populations.

CONCLUSION

Our results suggest that BWS is a reliable elicitation technique to value 3-level version of EQ-5D-Y health states in both adolescents and adults.

The acceptability of cervical electrical impedance spectroscopy within a multi-modal preterm birth screening package: a mixed methods study

BACKGROUND

Reducing the rate of preterm birth is a cornerstone of global efforts to address child mortality, however existing screening tests offer imperfect prediction. Cervical electrical impedance spectroscopy (EIS) is a novel technique to quantify the ripening changes which precede labour. Mid-trimester EIS measurements have been shown to accurately predict preterm birth in asymptomatic women. This study aimed to comprehensively evaluate the acceptability of cervical EIS to low and high-risk women as part of a package of screening tests performed during a larger prospective trial.

METHODS

In this parallel convergent mixed methods study, 40 women completed questionnaires before and after screening tests (EIS, cervical length measurement and fetal fibronectin quantification). Quantitative outcomes were anxiety levels before and after screening (Spielberger State Trait Anxiety Inventory, STAI-6), pain (Short Form McGill Pain Questionnaire) and ratings of EIS device appearance and test acceptability (visual analogue scales). Twenty-one women (11 high-risk, 10 low-risk) also attended a semi-structured qualitative interview. Interviews were recorded and transcribed, then thematic analysis was performed. A convergence coding matrix was constructed to enable triangulation of quantitative and qualitative results.

RESULTS

High risk women demonstrated a significant reduction in anxiety following screening (mean STAI-6 score 34.5 vs. 29.0, p = 0.002). A similar trend was observed among low-risk participants. Ratings of pain, EIS device appearance and procedural acceptability did not differ between groups. Mean pain ratings were low (visual analogue scale 0.97 and 1.01), comparing favourably to published evaluations of conventional screening tests. Qualitative analysis provided insight into both the physical consequences and emotional experiences of screening. Additional determinants of the screening experience included device design, pre-existing perspectives on intimate examination, attitudes to knowledge in pregnancy and interaction with clinical staff. Finally, a range of practical considerations regarding wider use of EIS were identified, with valuable complementary detail regarding acceptability for use in antenatal care.

CONCLUSIONS

Cervical EIS is well tolerated and acceptable to both low and high-risk women when performed as part of a multi-modal screening package. These results provide useful insights to inform the design of future study and screening protocols.

Discrete choice experiment with duration versus time trade-off: a comparison of test-retest reliability of health utility elicitation approaches in SF-6Dv2 valuation

OBJECTIVES

To evaluate and compare the test-retest reliability of discrete choice experiments with duration (DCE_TTO) and time trade-off (TTO) in the Chinese SF-6Dv2 valuation study.

METHODS

During face-to-face interviews, a representative sample of the Chinese general population completed 8 TTO tasks and 10 DCE_TTO tasks. Retest interviews were conducted after two weeks. For both DCE_TTO and TTO, the consistency of raw responses between the two tests was firstly evaluated at the individual level. Regressions were conducted to investigate the association between the test-retest reliability and the respondents' characteristics and the severity of health states. Consistency was then analyzed at the aggregate level by comparing the rank order of the coefficients of dimensions.

RESULTS

In total, 162 respondents (51.9% male; range 18-80 years) completed the two tests. The intraclass correlations coefficient 0.958 for TTO, with identical values accounting for 59.3% of observations. 76.4% of choices were identical for DCE_TTO, with a Kappa statistic of 0.528. Respondents' characteristics had no significant impact while the severity of health states valued in TTO and DCE_TTO tasks had a significant impact on the test-retest reliability. Both approaches produced relatively stable rank order of dimensions in constrained model estimations between test and retest data.

CONCLUSIONS

Individual responses of both approaches are relatively stable over time. The rank orders of dimensions in model estimations between test and retest for TTO and DCE_TTO are also consistent. The differences of utility estimation between the two tests for DCE_TTO need to be further investigated based on a larger sample size.

Severity-Stratified Discrete Choice Experiment Designs for Health State Evaluations

BACKGROUND

Discrete choice experiments (DCEs) are increasingly used for health state valuations. However, the values derived from initial DCE studies vary widely. We hypothesize that these findings indicate the presence of unknown sources of bias that must be recognized and minimized. Against this background, we studied whether values derived from a DCE are sensitive to how well the DCE design spans the severity range.

METHODS

We constructed an experiment involving three variants of DCE tasks for health state valuation: standard DCE, DCE-death, and DCE-duration. For each type of DCE, an experimental design was generated under two different conditions, enabling a comparison of health state values derived from current best practice Bayesian efficient DCE designs with values derived from 'severity-stratified' designs that control for coverage of the severity range in health state selection. About 3000 respondents participated in the study and were randomly assigned to one of the six study arms.

RESULTS

Imposing the severity-stratified restriction had a large effect on health states sampled for the DCE-duration approach. The unstratified efficient design returned a skewed distribution of selected health states, and this introduced bias. The choice probability of bad health states was underestimated, and time trade-offs to avoid bad states were overestimated, resulting in too low values. Imposing the same restriction had limited effect in the DCE-death approach and standard DCE.

CONCLUSION

Variation in DCE-derived values can be partially explained by differences in how well selected health states spanned the severity range. Imposing a 'severity stratification' on DCE-duration designs is a validity requirement.

A framework to address key issues of neonatal service configuration in England: the NeoNet multimethods study

Background

There is an inherent tension in neonatal services between the efficiency and specialised care that comes with centralisation and the provision of local services with associated ease of access and community benefits. This study builds on previous work in South West England to address these issues at a national scale.

Objectives

(1) To develop an analytical framework to address key issues of neonatal service configuration in England, (2) to investigate visualisation tools to facilitate the communication of findings to stakeholder groups and (3) to assess parental preferences in relation to service configuration alternatives.

Main outcome measures

The ability to meet nurse staffing guidelines, volumes of units, costs, mortality, number and distance of transfers, travel distances and travel times for parents.

Design

Descriptive statistics, location analysis, mathematical modelling, discrete event simulation and economic analysis were used. Qualitative methods were used to interview policy-makers and parents. A parent advisory group supported the study.

Setting

NHS neonatal services across England.

Data

Neonatal care data were sourced from the National Neonatal Research Database. Information on neonatal units was drawn from the National Neonatal Audit Programme. Geographic and demographic data were sourced from the Office for National Statistics. Travel time data were retrieved via a geographic information system. Birth data were sourced from Hospital Episode Statistics. Parental cost data were collected via a survey.

Results

Location analysis shows that to achieve 100% of births in units with ≥ 6000 births per year, the number of birth centres would need to be reduced from 161 to approximately 72, with more parents travelling > 30 minutes. The maximum number of neonatal intensive care units (NICUs) needed to achieve 100% of very low-birthweight infants attending high-volume units is 36 with existing NICUs, or 48 if NICUs are located wherever there is currently a neonatal unit of any level. Simulation modelling further demonstrated the workforce implications of different configurations. Mortality modelling shows that the birth of very preterm infants in high-volume hospitals reduces mortality (a conservative estimate of a 1.2-percentage-point lower risk) relative to these births in other hospitals. It is currently not possible to estimate the impact of mortality for infants transferred into NICUs. Cost modelling shows that the mean length of stay following a birth in a high-volume hospital is 9 days longer and the mean cost is £5715 more than for a birth in another neonatal unit. In addition, the incremental cost per neonatal life saved is £460,887, which is comparable to other similar life-saving interventions. The analysis of parent costs identified unpaid leave entitlement, food, travel, accommodation, baby care and parking as key factors. The qualitative study suggested that central concerns were the health of the baby and mother, communication by medical teams and support for families.

Limitations

The following factors could not be modelled because of a paucity of data – morbidity outcomes, the impact of transfers and the maternity/neonatal service interface.

Conclusions

An evidence-based framework was developed to inform the configuration of neonatal services and model system performance from the perspectives of both service providers and parents.

Future work

To extend the modelling to encompass the interface between maternity and neonatal services.

Funding

The National Institute for Health Research Health Services and Delivery Research programme.

Measuring health utility in varying pregnancy contexts among a diverse cohort of pregnant women

OBJECTIVE

To contribute to decision analysis by estimating utility, defined as an individual's valuation of specific health states, for different pregnancy contexts.

STUDY DESIGN

Cross-sectional analysis of data from pregnant women recruited at pregnancy testing clinics during June 2014-June 2015. Utility was measured using the visual analog scale (VAS), PROMIS GSF-derived utility, standard gamble (SG), and time-trade-off (TTO) approaches. Six dimensions of pregnancy context were assessed including: intention, desirability, planning, timing, wantedness, and happiness. Multivariable regression modeling was used to examine the associations between pregnancy context and utility while controlling for women's sociodemographic and health characteristics.

RESULTS

Among 123 participants with diverse characteristics, aged 27±6 years, with mean gestation of 7.5±3 weeks, few reported optimal pregnancy contexts. Mean utility of the pregnancy state varied across contexts, whether measured with VAS (0.28-0.91), PROMIS GSF-derived utility (0.66-0.75), SG (0.985-1.00) or TTO (0.9990-0.99999). The VAS-derived mean utility score for unintended pregnancy was 0.68 (95% CI 0.59, 0.77). Multivariable regression analysis demonstrated significant disutility of unintended pregnancy, as well as all other unfavorable pregnancy contexts, when measured by VAS. In contrast, PROMIS GSF-derived utility only detected a significant reduction in utility among ambivalent compared to wanted pregnancy, while SG and TTO did not show meaningful differences in utility across pregnancy contexts.

CONCLUSIONS

Unintended pregnancy is associated with significant patient-reported disutility, as is pregnancy occurring in other unfavorable contexts. VAS-based measurements provide the most nuanced measures of the utility for pregnancy in varying contexts.

IMPLICATIONS

Decision analyses, including assessments of the cost-effectiveness of pregnancy related interventions, should incorporate measures of the utility of pregnancy in various contexts.

Head-to-head comparison of health-state values derived by a probabilistic choice model and scores on a visual analogue scale

BACKGROUND

Health states were quantified based on discrete choice (DC) modeling and visual analogue scale (VAS) values using the five-level version of the EQ-5D (EQ-5D-5L). The aim of this study was to determine the extent of the relationship between DC derived values (indirect method) and VAS values (direct method).

METHODS

Data were collected in Canada, the United Kingdom, the Netherlands, and the United States. Respondents were asked to perform paired comparisons between two EQ-5D-5L health states for DC. In total, 400 different EQ-5D-5L states were included. After each DC task, respondents were prompted to score the two states one after another on a VAS. Intraclass correlation coefficients were calculated between DC and VAS values and illuminating graphs were designed.

RESULTS

Approximately 400 respondents participated from each country. High similarity [individual intraclass correlation coefficients (ICC) >0.85] of DC and moderate correspondence of VAS values were observed for the four countries. Cross-country comparison of DC values shows a nonlinear relationship to the VAS values.

CONCLUSION

EQ-5D-5L derived DC and VAS values show a close but nonlinear relationship. Given the obvious biases associated with the VAS, DC methods based on ordinal responses may be a better alternative.

Planned home compared with planned hospital births: mode of delivery and Perinatal mortality rates, an observational study

Background

To compare the mode of delivery between planned home versus planned hospital births and to determine if differences in intervention rates could be interpreted as over- or undertreatment.

Methods

Intervention and perinatal mortality rates were obtained for 679,952 low-risk women from the Dutch Perinatal Registry (2000–2007). Intervention was defined as operative vaginal delivery and/or caesarean section. Perinatal mortality was defined as the intrapartum and early neonatal mortality rate up to 7 days postpartum.

Besides adjustment for maternal and care factors, we included for additional casemix adjustment: presence of congenital abnormality, small for gestational age, preterm birth, or low Apgar score. The techniques used were nested multiple stepwise logistic regression, and stratified analysis for separate risk groups. An intention-to-treat like analysis was performed.

Results

The intervention rate was lower in planned home compared to planned hospital births (10.9% 95% CI 10.8–11.0 vs. 13.8% 95% CI 13.6–13.9). Intended place of birth had significant impact on the likelihood to intervene after adjustment (planned homebirth (OR 0.77 95% CI. 0.75–0.78)).

The mortality rate was lower in planned home births (0.15% vs. 0.18%). After adjustment, the interaction term home- intervention was significant (OR1.51 95% CI 1.25–1.84). In risk groups, a higher perinatal mortality rate was observed in planned home births.

Conclusions

The potential presence of over- or under treatment as expressed by adjusted perinatal mortality differs per risk group. In planned home births especially multiparous women showed universally lower intervention rates. However, the benefit of substantially fewer interventions in the planned home group seems to be counterbalanced by substantially increased mortality if intervention occurs.

Valuing Meta-Health Effects for Use in Economic Evaluations to Inform Reimbursement Decisions: A Review of the Evidence

OBJECTIVE

This review explores the evidence from the literature regarding how meta-health effects (effects other than health resulting from the consumption of health care) are valued for use in economic evaluations.

METHODS

A systematic review of the published literature (the EMBASE, MEDLINE, PsycINFO, CINAHL, EconLit and SocINDEX databases were searched for publications in March 2016, plus manual searching) investigated the associations between study methods and the resulting values for meta-health effects estimated for use in economic evaluations. The review considered which meta-health effects were being valued and how this differed by evaluation approach, intervention investigated, source of funds and year of publication. Detailed reasons for differences observed between values for comparable meta-health effects were explored, accounting for the method of valuation.

RESULTS

The search of the literature revealed 71 studies of interest; 35% involved drug interventions, with convenience, information and process of care the three meta-health effects most often investigated. Key associations with the meta-health effects were the evaluation method, the intervention, and the source of funds. Relative values for meta-health effects ranged from 0.9% to 68% of the overall value reported in a study. For a given meta-health effect, the magnitude of the effect evaluated and how the meta-health effect was described and framed relative to overall health explained the differences in relative values.

CONCLUSIONS

Evidence from the literature shows variability in how meta-health effects are being measured for use in economic evaluations. Understanding the sources of that variability is important if decision makers are to have confidence in how meta-health effects are valued.

Different settings of place of midwife-led birth: evaluation of a midwife-led birth centre

Objectives

The claimed advantages of home deliveries, including fewer medical interventions, are potentially counter balanced by the small additional risk on perinatal adverse outcome compared to hospital deliveries in low risk women. Homelike birth centres have been proposed a new setting for low risk women combining the advantages of home and hospital, resulting in lower intervention rates with equal safety. This paper addresses whether the introduction of a midwife-led birth centre adjacent to the hospital combines the advantages of home and hospital deliveries. Additionally, we investigate whether the introduction of a midwife-led birth centre leads to a different risk selection of women planning their delivery either at home, at the hospital or at the birth centre.

Methods

Anonymized data, between January 2007 and June 2012, was collected from the four participating midwife practices. Women (n = 5558) were categorized according to intended place of birth. Women’s characteristics and pregnancy outcomes were compared between the period before and after its introduction using Chi square and Fisher’s Exact tests. Direct and indirect standardized rates were calculated for different outcomes [(1) intrapartum and neonatal mortality (<24 h), (2) composite outcome of neonatal morbidities, (3) composite outcome of maternal morbidities, and (4) medical intervention], taking the period before introduction as reference.

Results

After the introduction of the birth centre a different risk selection was observed. Women’s characteristics were most unfavourable for intended birth centre births. Additionally, an higher neonatal risk load was seen within these women. After its introduction neonatal morbidities decreased (5.0 vs. 3.8 %) and maternal morbidities decreased (8.3 vs. 7.3 %). Interventions were about equal. Direct and indirect standardization provided similar results.

Conclusion

Neonatal morbidity and maternal morbidity tended to decrease, while overall intervention rates were unaffected. The introduction of the midwife-led birth centre seems to benefit the outcome of midwife-led deliveries. We interpret this change by the redistribution of the higher risk women among the low risk population intending birth at the birth centre instead of home.

Valuing Treatments for Parkinson Disease Incorporating Process Utility: Performance of Best-Worst Scaling, Time Trade-Off, and Visual Analogue Scales

OBJECTIVE

The objective of this study was to compare treatment profiles including both health outcomes and process characteristics in Parkinson disease using best-worst scaling (BWS), time trade-off (TTO), and visual analogue scales (VAS).

METHODS

From the model comprising of seven attributes with three levels, six unique profiles were selected representing process-related factors and health outcomes in Parkinson disease. A Web-based survey (N = 613) was conducted in a general population to estimate process-related utilities using profile-based BWS (case 2), multiprofile-based BWS (case 3), TTO, and VAS. The rank order of the six profiles was compared, convergent validity among methods was assessed, and individual analysis focused on the differentiation between pairs of profiles with methods used.

RESULTS

The aggregated health-state utilities for the six treatment profiles were highly comparable for all methods and no rank reversals were identified. On the individual level, the convergent validity between all methods was strong; however, respondents differentiated less in the utility of closely related treatment profiles with a VAS or TTO than with BWS. For TTO and VAS, this resulted in nonsignificant differences in mean utilities for closely related treatment profiles.

CONCLUSIONS

This study suggests that all methods are equally able to measure process-related utility when the aim is to estimate the overall value of treatments. On an individual level, such as in shared decision making, BWS allows for better prioritization of treatment alternatives, especially if they are closely related. The decision-making problem and the need for explicit trade-off between attributes should determine the choice for a method.

The better than dead method: feasibility and interpretation of a valuation study

BACKGROUND

Traditionally, the valuation of health states worse than being dead suffers from two problems: [1] the use of different elicitation methods for positive and negative values, necessitating arbitrary transformations to map negative to positive values; and [2] the inability to quantify that values are time dependent. The Better than Dead (BTD) method is a health-state valuation method where states with a certain duration are compared with being dead. It has the potential to overcome these problems.

OBJECTIVES

To test the feasibility of the BTD method to estimate values for the EQ-5D system.

METHODS

A representative sample of 291 Dutch respondents (aged 18-45 years) was recruited. In a web-based questionnaire, preferences were elicited for a selection of 50 different health states with six durations between 1 and 40 years. Random-effects models were used to estimate the effects of socio-demographic and experimental variables, and to estimate values for the EQ-5D. Test-retest reliability was assessed in 41 respondents.

RESULTS

Important determinants for BTD were a religious life stance [odds ratio 4.09 (2.00-8.36)] and the educational level. The fastest respondents more often preferred health-state scenarios to being dead and had lower test-retest reliability (0.45 versus 0.77 and 0.84 for fast, medium and slow response times, respectively). The results showed a small number of so-called maximal endurable time states.

CONCLUSION

Valuating health states using the BTD method is feasible and reliable. Further research should explore how the experimental setting modifies how values depend on time.

Obstetrical outcome valuations by patients, professionals, and laypersons: differences within and between groups using three valuation methods

Background

Decision-making can be based on treatment preferences of the patient, the doctor, or by guidelines based on lay people's preferences. We compared valuations assigned by three groups: patients, obstetrical care professionals, and laypersons, for health states involving both mother and (unborn) child. Our aim was to compare the valuations of different groups using different valuation methods and complex obstetric health outcome vignettes that involve both maternal and neonatal outcomes.

Methods

Patients (n = 24), professionals (n = 30), and laypersons (n = 27) valued the vignettes using three valuation methods: visual analogue scale (VAS), time trade-off (TTO), and discrete choice experimentation (DCE). Each vignette covered five health attributes: maternal health ante partum, time between diagnosis and delivery, process of delivery, maternal outcome, and neonatal outcome. We used feasibility questionnaires, Generalization theory, test-retest reliability and within-group reliability to compare the valuation patterns between groups and methods. We assessed relative weights from each valuation method to test for consistency across groups.

Results

Test-retest reliability was equal across groups, but different across methods: highest for VAS (ICC = 0.61-0.73), intermediate for TTO (ICC = 0.24-0.74) and lowest for DCE (kappa = 0.15-0.37). Within-group reliability was highest in all groups with VAS (ICC = 0.70-0.73), intermediate with DCE (kappa = 0.56-0.76) and lowest with TTO (ICC = 0.20-0.66). Effects of groups were smaller than effects of methods. Differences between groups were largest for severe health states.

Conclusion

Based on our results, decision making among laypersons should use TTO or DCE; patients should use VAS or TTO.

Eliciting willingness to pay in obstetrics: comparing a direct and an indirect valuation method for complex health outcomes

OBJECTIVE

To compare direct and indirect willingness to pay (WTP) elicitation methods in terms of feasibility, reliability, and comparability. The application is obstetrics, where always both a mother's and a child's health are at stake.

METHODS

An open-ended contingent valuation method (CVM) as a direct WTP elicitation method, and the discrete choice experiment (DCE) as an indirect WTP elicitation method. Vignettes to be valued were based on clinical patient data. Participants were 88 laypersons who received their questionnaires by postal mail.

RESULTS

The DCE task was completed faster (p=0.006) and was regarded easier (p<0.001) than the CVM task. Test-retest for CVM was substantial (ICC=0.76), and for DCE moderate (k=0.49). Female sex (p<0.001), age≥50 years (p=0.013), higher income (p<0.001), and higher education (p<0.001) were associated with higher WTP. Correlation between CVM and DCE was 0.79 (Kendall's Tau-b; p<0.001). The implied WTP as derived with DCE was between 2.3 and 10.2 times higher than with CVM. The relationship between the WTPs was linear.

CONCLUSION

It is yet unclear what lies behind the numbers of DCE. DCE has no methodological benefits over the conventional CVM when eliciting WTP for complex health outcomes in obstetrics.