Constructing indirect utility models: some observations on the principles and practice of mapping to obtain health state utilities

The Patient Experience of Thyroid Cancer

The 3 phases of thyroid cancer care are discussed: diagnosis, management, and survivorship. Drivers of quality of life (QOL) in each phase are described, and suggestions are made for mitigating the risk of poor QOL. Active surveillance is another emerging management strategy that has the potential to improve QOL by eliminating upfront surgical morbidity but will need to be studied prospectively.

A video-feedback parenting intervention to prevent enduring behaviour problems in at-risk children aged 12-36 months: the Healthy Start, Happy Start RCT

BACKGROUND

Behaviour problems emerge early in childhood and place children at risk for later psychopathology.

OBJECTIVES

To evaluate the clinical effectiveness and cost-effectiveness of a parenting intervention to prevent enduring behaviour problems in young children.

DESIGN

A pragmatic, assessor-blinded, multisite, two-arm, parallel-group randomised controlled trial.

SETTING

Health visiting services in six NHS trusts in England.

PARTICIPANTS

A total of 300 at-risk children aged 12-36 months and their parents/caregivers.

INTERVENTIONS

Families were allocated in a 1 : 1 ratio to six sessions of Video-feedback Intervention to promote Positive Parenting and Sensitive Discipline (VIPP-SD) plus usual care or usual care alone.

MAIN OUTCOME MEASURES

The primary outcome was the Preschool Parental Account of Children's Symptoms, which is a structured interview of behaviour symptoms. Secondary outcomes included caregiver-reported total problems on the Child Behaviour Checklist and the Strengths and Difficulties Questionnaire. The intervention effect was estimated using linear regression. Health and social care service use was recorded using the Child and Adolescent Service Use Schedule and cost-effectiveness was explored using the Preschool Parental Account of Children's Symptoms.

RESULTS

In total, 300 families were randomised: 151 to VIPP-SD plus usual care and 149 to usual care alone. Follow-up data were available for 286 (VIPP-SD, n = 140; usual care, n = 146) participants and 282 (VIPP-SD, n = 140; usual care, n = 142) participants at 5 and 24 months, respectively. At the post-treatment (primary outcome) follow-up, a group difference of 2.03 on Preschool Parental Account of Children's Symptoms (95% confidence interval 0.06 to 4.01; p = 0.04) indicated a positive treatment effect on behaviour problems (Cohen's d = 0.20, 95% confidence interval 0.01 to 0.40). The effect was strongest for children's conduct [1.61, 95% confidence interval 0.44 to 2.78; p = 0.007 (d = 0.30, 95% confidence interval 0.08 to 0.51)] versus attention deficit hyperactivity disorder symptoms [0.29, 95% confidence interval -1.06 to 1.65; p = 0.67 (d = 0.05, 95% confidence interval -0.17 to 0.27)]. The Child Behaviour Checklist [3.24, 95% confidence interval -0.06 to 6.54; p = 0.05 (d = 0.15, 95% confidence interval 0.00 to 0.31)] and the Strengths and Difficulties Questionnaire [0.93, 95% confidence interval -0.03 to 1.9; p = 0.06 (d = 0.18, 95% confidence interval -0.01 to 0.36)] demonstrated similar positive treatment effects to those found for the Preschool Parental Account of Children's Symptoms. At 24 months, the group difference on the Preschool Parental Account of Children's Symptoms was 1.73 [95% confidence interval -0.24 to 3.71; p = 0.08 (d = 0.17, 95% confidence interval -0.02 to 0.37)]; the effect remained strongest for conduct [1.07, 95% confidence interval -0.06 to 2.20; p = 0.06 (d = 0.20, 95% confidence interval -0.01 to 0.42)] versus attention deficit hyperactivity disorder symptoms [0.62, 95% confidence interval -0.60 to 1.84; p = 0.32 (d = 0.10, 95% confidence interval -0.10 to 0.30)], with little evidence of an effect on the Child Behaviour Checklist and the Strengths and Difficulties Questionnaire. The primary economic analysis showed better outcomes in the VIPP-SD group at 24 months, but also higher costs than the usual-care group (adjusted mean difference £1450, 95% confidence interval £619 to £2281). No treatment- or trial-related adverse events were reported. The probability of VIPP-SD being cost-effective compared with usual care at the 24-month follow-up increased as willingness to pay for improvements on the Preschool Parental Account of Children's Symptoms increased, with VIPP-SD having the higher probability of being cost-effective at willingness-to-pay values above £800 per 1-point improvement on the Preschool Parental Account of Children's Symptoms.

LIMITATIONS

The proportion of participants with graduate-level qualifications was higher than among the general public.

CONCLUSIONS

VIPP-SD is effective in reducing behaviour problems in young children when delivered by health visiting teams. Most of the effect of VIPP-SD appears to be retained over 24 months. However, we can be less certain about its value for money.

TRIAL REGISTRATION

Current Controlled Trials ISRCTN58327365.

FUNDING

This project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 25, No. 29. See the NIHR Journals Library website for further project information.

Preference-based measures of health-related quality of life in congenital mobility impairment: a systematic review of validity and responsiveness

INTRODUCTION

Mobility impairment is the leading cause of disability in the UK. Individuals with congenital mobility impairments have unique experiences of health, quality of life and adaptation. Preference-based outcomes measures are often used to help inform decisions about healthcare funding and prioritisation, however the applicability and accuracy of these measures in the context of congenital mobility impairment is unclear. Inaccurate outcome measures could potentially affect the care provided to these patient groups. The aim of this systematic review was to examine the performance of preference-based outcome measures for the measurement of utility values in various forms of congenital mobility impairment.

METHODS

Ten databases were searched, including Science Direct, CINAHL and PubMed. Screening of reference lists and hand-searching were also undertaken. Descriptive and narrative syntheses were conducted to combine and analyse the various findings. Results were grouped by condition. Outcome measure performance indicators were adapted from COSMIN guidance and were grouped into three broad categories: validity, responsiveness and reliability. Screening, data extraction and quality appraisal were carried out by two independent reviewers.

RESULTS

A total of 31 studies were considered eligible for inclusion in the systematic review. The vast majority of studies related to either cerebral palsy, spina bifida or childhood hydrocephalus. Other relevant conditions included muscular dystrophy, spinal muscular atrophy and congenital clubfoot. The most commonly used preference-based outcome measure was the HUI3. Reporting of performance properties predominantly centred around construct validity, through known group analyses and assessment of convergent validity between comparable measures and different types of respondents. A small number of studies assessed responsiveness, but assessment of reliability was not reported. Increased clinical severity appears to be associated with decreased utility outcomes in congenital mobility impairment, particularly in terms of gross motor function in cerebral palsy and lesion level in spina bifida. However, preference-based measures exhibit limited correlation with various other condition-specific and clinically relevant outcome measures.

CONCLUSION

Preference-based measures exhibit important issues and discrepancies relating to validity and responsiveness in the context of congenital mobility impairment, thus care must be taken when utilising these measures in conditions associated with congenital mobility impairments.

Using the Fatigue Severity Scale to inform healthcare decision-making in multiple sclerosis: mapping to three quality-adjusted life-year measures (EQ-5D-3L, SF-6D, MSIS-8D)

Background

Fatigue has a major influence on the quality of life of people with multiple sclerosis. The Fatigue Severity Scale is a frequently used patient-reported measure of fatigue impact, but does not generate the health state utility values required to inform cost-effectiveness analysis, limiting its applicability within decision-making contexts. The objective of this study was to use statistical mapping methods to convert Fatigue Severity Scale scores to health state utility values from three preference-based measures: the EQ-5D-3L, SF-6D and Multiple Sclerosis Impact Scale-8D.

Methods

The relationships between the measures were estimated through regression analysis using cohort data from 1056 people with multiple sclerosis in South West England. Estimation errors were assessed and predictive performance of the best models as tested in a separate sample (n = 352).

Results

For the EQ-5D and the Multiple Sclerosis Impact Scale-8D, the best performing models used a censored least absolute deviation specification, with Fatigue Severity Scale total score, age and gender as predictors. For the SF-6D, the best performing model used an ordinary least squares specification, with Fatigue Severity Scale total score as the only predictor.

Conclusions

Here we present algorithms to convert Fatigue Severity Scales scores to health state utility values based on three preference-based measures. These values may be used to estimate quality-adjusted life-years for use in cost-effectiveness analyses and to consider the health-related quality of life of people with multiple sclerosis, thereby informing health policy decisions.

Electronic supplementary material

The online version of this article (10.1186/s12955-019-1205-y) contains supplementary material, which is available to authorized users.

Training teachers in classroom management to improve mental health in primary school children: the STARS cluster RCT

Background

Poor mental health in childhood is common, persistent and associated with a range of adverse outcomes that include persistent psychopathology, as well as risk-taking behaviour, criminality and educational failure, all of which may also compromise health. There is a growing policy focus on children’s mental health and the role of schools in particular in addressing this.

Objectives

To evaluate whether or not the Incredible Years®(IY) Teacher Classroom Management (TCM) training improved children’s mental health, behaviour, educational attainment and enjoyment of school, improved teachers’ mental health and relationship with work, and was cost-effective in relation to potential improvements.

Design

A two-arm, pragmatic, parallel-group, superiority, cluster randomised controlled trial.

Setting

A total of 80 UK schools (clusters) were recruited in three distinct cohorts between 2012 and 2014 and randomised to TCM (intervention) or teaching as usual [(TAU) control] with follow-ups at 9, 18 and 30 months. Schools and teachers were not masked to allocation.

Participants

Eighty schools (n = 2075 children) were randomised: 40 (n = 1037 children) to TCM and 40 (n = 1038 children) to TAU.

Interventions

TCM was delivered to teachers in six whole-day sessions, spread over 6 months. The explicit goals of TCM are to enhance classroom management skills and improve teacher–student relationships.

Main outcome measures

The primary planned outcome was the teacher-reported Strengths and Difficulties Questionnaire Total Difficulties (SDQ-TD) score. Random-effects linear regression and marginal logistic regression models using generalized estimating equations were used to analyse outcomes.

Results

The intervention reduced the SDQ-TD score at 9 months [adjusted mean difference (AMD) –1.0, 95% confidence interval (CI) –1.9 to –0.1;p = 0.03] but there was little evidence of effects at 18 months (AMD –0.1, 95% CI –1.5 to 1.2;p = 0.85) and 30 months (AMD –0.7, 95% CI –1.9 to 0.4;p = 0.23). Planned subgroup analyses suggested that TCM is more effective than TAU for children with poor mental health. Cost-effectiveness analysis using the SDQ-TD suggested that the probability of TCM being cost-effective compared with TAU was associated with some uncertainty (range of 40% to 80% depending on the willingness to pay for a unit improvement in SDQ-TD score). In terms of quality-adjusted life-years (QALYs), there was evidence to suggest that TCM was cost-effective compared with TAU at the National Institute for Health and Care Excellence thresholds of £20,000–30,000 per QALY at 9- and 18-month follow-up, but not at 30-month follow-up. There was evidence of reduced disruptive behaviour (p = 0.04) and reductions in inattention and overactivity (p = 0.02) at the 30-month follow-up. Despite no main effect on educational attainment, subgroup analysis indicated that the intervention’s effect differed between those who did and those who did not have poor mental health for both literacy (interactionp = 0.04) and numeracy (interactionp = 0.03). Independent blind observations and qualitative feedback from teachers suggested that teachers’ behaviour in the classroom changed as a result of attending TCM training.

Limitations

Teachers were not masked to allocation and attrition was marked for parent-reported data.

Conclusions

Our findings provide tentative evidence that TCM may be an effective universal child mental health intervention in the short term, particularly for primary school children who are identified as struggling, and it may be a cost-effective intervention in the short term.

Future work

Further research should explore TCM as a whole-school approach by training all school staff and should evaluate the impact of TCM on academic progress in a more thorough and systematic manner.

Trial registration

Current Controlled Trials ISRCTN84130388.

Funding

This project was funded by the National Institute for Health Research (NIHR) Public Health Research programme and will be published in full inPublic Health Research; Vol. 7, No. 6. See the NIHR Journals Library website for further project information. Funding was also provided by the NIHR Collaboration for Leadership in Applied Health Research and Care South West Peninsula (NIHR CLAHRC South West Peninsula).

Use of Bayesian methods to model the SF-6D health state preference based data

Background

Conventionally, models used for health state valuation data have been frequentists. Recently a number of researchers have investigated the use of Bayesian methods in this area. The aim of this paper is to put on the map of modelling a new approach to estimating SF-6D health state utility values using Bayesian methods. This will help health care professionals in deriving better health state utilities of the original UK SF-6D for their specialized applications.

Methods

The valuation study is composed of 249 SF-6D health states valued by a representative sample of the UK population using the standard gamble technique. Throughout this paper, we present four different models, including one simple linear regression model and three random effect models. The predictive ability of these models is assessed by comparing predicted and observed mean SF-6D scores, R²/adjusted R² and RMSE. All analyses were carried out using Bayesian Markov chain Monte Carlo (MCMC) simulation methods freely available in the specialist software WinBUGS.

Results

The random effects model with interaction model performs best under all criterions, with mean predicted error of 0.166, R²/adjusted R² of 0.683 and RMSE of 0.218.

Conclusions

The Bayesian models provide flexible approaches to estimate mean SF-6D utility estimates, including characterizing the full range of uncertainty inherent in these estimates. We hope that this work will provide applied researchers with a practical set of tools to appropriately model outcomes in cost-effectiveness analysis.

Development of Conversion Functions Mapping the FACT-B Total Score to the EQ-5D-5L Utility Value by Three Linking Methods and Comparison with the Ordinary Least Square Method

INTRODUCTION

Health-related quality-of-life (HRQoL) measures are commonly mapped to a value that represents a utility for economic evaluation via regression models, which may lead to shrinkage of the variance.

OBJECTIVES

This study aimed to develop and compare conversion functions that map the Functional Assessment of Cancer Therapy-Breast (FACT-B) total score to the EuroQoL 5-Dimensions, 5-Levels (EQ-5D-5L) utility value via four methods.

METHODS

We used the HRQoL scores of 238 Singapore patients with breast cancer to develop the conversion function for the equipercentile, linear equating, mean rank and ordinary least squares (OLS) methods. We compared the distributions of the observed values and the four sets of mapped values and performed regression analyses to assess whether the association with risk factors was preserved by utility values derived from mapping.

RESULTS

At baseline, the observed EQ-5D-5L utility value had a mean ± standard deviation (SD) of 0.820 ± 0.152, and 24.8% of the respondents attained a value of 1. The OLS method (mean 0.820; SD 0.112; proportion 0%) better agreed with the observed data than the equipercentile (mean 0.831; SD 0.152; proportion 23.5%), linear equating (mean 0.814; SD 0.145; proportion 11.8%) and mean rank method (mean 0.821; SD 0.147; proportion 23.9%). The significance of association was preserved for all parameters involved in the regression analyses by the equipercentile and linear equating methods, but the mean rank and OLS methods were inconsistent with the observed data for one and two parameters, respectively.

CONCLUSION

The problem of shrinkage in the variance occurred in the OLS method, but it provided an unbiased estimate for the mean and better agreement. Among the other three linking methods, the mean rank method better described the distribution, whereas the equipercentile and linear equating methods better assessed the association with risk factors.

Statistical Alchemy: Conceptual Validity and Mapping to Generate Health State Utility Values

Mapping between non-preference- and preference-based health-related quality-of-life instruments has become a common technique for estimating health state utility values for use in economic evaluations. Despite the increased use of mapped health state utility estimates in health technology assessment and economic evaluation, the methods for deriving them have not been fully justified. Recent guidelines aim to standardise reporting of the methods used to map between instruments but do not address fundamental concerns in the underlying conceptual model. Current mapping methods ignore the important conceptual issues that arise when extrapolating results from potentially unrelated measures. At the crux of the mapping problem is a question of validity; because one instrument can be used to predict the scores on another, does this mean that the same preference for health is being measured in actual and estimated health state utility values? We refer to this as conceptual validity. This paper aims to (1) explain the idea of conceptual validity in mapping and its implications; (2) consider the consequences of poor conceptual validity when mapping for decision making in the context of healthcare resource allocation; and (3) offer some preliminary suggestions for improving conceptual validity in mapping.

Mapping the EORTC QLQ-C30 onto the EQ-5D-3L: assessing the external validity of existing mapping algorithms

PURPOSE

To determine the external validity of existing mapping algorithms for predicting EQ-5D-3L utility values from EORTC QLQ-C30 responses and to establish their generalizability in different types of cancer.

METHODS

A main analysis (pooled) sample of 3560 observations (1727 patients) and two disease severity patient samples (496 and 93 patients) with repeated observations over time from Cancer 2015 were used to validate the existing algorithms. Errors were calculated between observed and predicted EQ-5D-3L utility values using a single pooled sample and ten pooled tumour type-specific samples. Predictive accuracy was assessed using mean absolute error (MAE) and standardized root-mean-squared error (RMSE). The association between observed and predicted EQ-5D utility values and other covariates across the distribution was tested using quantile regression. Quality-adjusted life years (QALYs) were calculated using observed and predicted values to test responsiveness.

RESULTS

Ten 'preferred' mapping algorithms were identified. Two algorithms estimated via response mapping and ordinary least-squares regression using dummy variables performed well on number of validation criteria, including accurate prediction of the best and worst QLQ-C30 health states, predicted values within the EQ-5D tariff range, relatively small MAEs and RMSEs, and minimal differences between estimated QALYs. Comparison of predictive accuracy across ten tumour type-specific samples highlighted that algorithms are relatively insensitive to grouping by tumour type and affected more by differences in disease severity.

CONCLUSIONS

Two of the 'preferred' mapping algorithms suggest more accurate predictions, but limitations exist. We recommend extensive scenario analyses if mapped utilities are used in cost-utility analyses.

Pressure UlceR Programme Of reSEarch (PURPOSE): using mixed methods (systematic reviews, prospective cohort, case study, consensus and psychometrics) to identify patient and organisational risk, develop a risk assessment tool and patient-reported outcome Quality of Life and Health Utility measures

Background

The Pressure UlceR Programme Of reSEarch (PURPOSE) consisted of two themes. Theme 1 focused on improving our understanding of individuals’ and organisational risk factors and on improving the quality of risk assessments (work packages 1–3) and theme 2 focused on developing patient-reported outcome measures (work packages 4 and 5).

Methods

The programme comprised 21 individual pieces of work. Pain: (1) multicentre pain prevalence study in acute hospitals, (2) multicentre pain prevalence study in community localities incorporating (3) a comparison of case-finding methods, and (4) multicentre, prospective cohort study. Severe pressure ulcers: (5) retrospective case study, (6) patient involvement workshop with the Pressure Ulcer Research Service User Network for the UK (PURSUN UK) and (7) development of root cause analysis methodology. Risk assessment: (8) systematic review, (9) consensus study, (10) conceptual framework development and theoretical causal pathway, (11) design and pretesting of draft Risk Assessment Framework and (12) field test to assess reliability, validity, data completeness and clinical usability. Quality of life: (13) conceptual framework development (systematic review, patient interviews), (14 and 15) provisional instrument development, with items generated from patient interviews [from (1) above] two systematic reviews and experts, (16) pretesting of the provisional Pressure Ulcer Quality of Life (PU-QOL) instrument using mixed methods, (17) field test 1 including (18) optimal mode of administration substudy and item reduction with testing of scale formation, acceptability, scaling assumptions, reliability and validity, and (19) field test 2 – final psychometric evaluation to test scale targeting, item response categories, item fit, response bias, acceptability, scaling assumptions, reliability and validity. Cost–utility: (20) time trade-off task valuations of health states derived from selected PU-QOL items, and (21) validation of the items selected and psychometric properties of the new Pressure Ulcer Quality of Life Utility Index (PUQOL-UI).

Key findings

Pain: prevalence studies – hospital and community patients experience both pressure area-related and pressure ulcer pain; pain cohort study – indicates that pain is independently predictive of category 2 (and above) pressure ulcer development. Severe pressure ulcers: these were more likely to develop in contexts in which clinicians failed to listen to patients/carers or recognise/respond to high risk or the presence of an existing pressure ulcer and services were not effectively co-ordinated; service users found the interactive workshop format valuable; including novel components (interviews with patients and carers) in root cause analysis improves the quality of the insights captured. Risk assessment: we developed a Pressure Ulcer Risk Assessment Framework, the PURPOSE-T, incorporating the Minimum Data Set, a screening stage, a full assessment stage, use of colour to support decision-making, and decision pathways that make a clear distinction between patients with an existing pressure ulcer(s) (or scarring from previous ulcers) who require secondary prevention and treatment and those at risk who require primary prevention (http://medhealth.leeds.ac.uk/accesspurposet). Quality of life: the final PU-QOL instrument consists of 10 scales to measure pain, exudate, odour, sleep, vitality, mobility/movement, daily activities, emotional well-being, self-consciousness and appearance, and participation (http://medhealth.leeds.ac.uk/puqol-ques). Cost–utility: seven items were selected from the PU-QOL instrument for inclusion in the PUQOL-UI (http://medhealth.leeds.ac.uk/puqol-ui); secondary study analysis indicated that item selection for the PUQOL-UI was appropriate and that the index was acceptable to patients and had adequate levels of validity.

Conclusions

The PURPOSE programme has provided important insights for pressure ulcer prevention and treatment and involvement of service users in research and development, with implications for patient and public involvement, clinical practice, quality/safety/health service management and research including replication of the pain risk factor study, work exploring ‘best practice’ settings, the impact of including skin status as an indicator for escalation of preventative interventions, further psychometric evaluation of PU-QOL and PUQOL-UI the measurement of ‘disease attribution.’

Funding

The National Institute for Health Research Programme Grants for Applied Research programme.

Use of Bayesian Markov Chain Monte Carlo Methods to Estimate EQ-5D Utility Scores from EORTC QLQ Data in Myeloma for Use in Cost-Effectiveness Analysis

Background. Patient-reported outcome measures are an important component of the evidence for health technology appraisal. Their incorporation into cost-effectiveness analyses (CEAs) requires conversion of descriptive information into utilities. This can be done by using bespoke utility algorithms. Otherwise, investigators will often estimate indirect utility models for the patient-reported outcome measures using off-the-shelf utility data such as the EQ-5D or SF-6D. Numerous modeling strategies are reported; however, to date, there has been limited utilization of Bayesian methods in this context. In this article, we examine the relative advantage of the Bayesian methods in relation to dealing with missing data, relaxing the assumption of equal variances and characterizing the uncertainty in the model predictions. Methods. Data from a large myeloma trial were used to examine the relationship between scores in each of the 19 domains of the European Organisation for Research and Treatment of Cancer (EORTC) QLQ-C30/QLQ-MY20 and the EQ-5D utility. Data from 1839 patients were divided 75%/25% between derivation and validation sets. A conventional ordinary least squares model assuming equal variance and a Bayesian model allowing unequal variance were estimated on complete cases. Two further models were estimated using conventional and Bayesian multiple imputation, respectively, using the full data set. Models were compared in terms of data fit, accuracy in model prediction, and characterization of uncertainty in model predictions. Conclusions. Mean EQ-5D utility weights can be estimated from the EORTC QLQ-C30/QLQ-MY20 for use in CEAs. Frequentist and Bayesian methods produced effectively identical models. However, the Bayesian models provide distributions describing the uncertainty surrounding the estimated utility values and are thus more suited informing analyses for probabilistic CEAs.

Novel methods of measuring clinical outcomes from psoriasis and psoriatic arthritis clinical trials

Numerous instruments exist that measure the clinical and health related quality of life impact of psoriasis and psoriatic arthritis (PsA) in clinical trials. However, many of these instruments are not typically used in economic evaluations to inform decision problems facing health care decision makers. This study reviews the current state of psoriasis and PsA health outcome measures and evaluates their limitations in cost-effectiveness analyses (CEAs). We highlight the health related quality of life and clinical outcome measures that are typically used in CEAs, with special focus on studies with quality adjusted life years as a primary outcome measure. Despite the high prevalence of psoriasis and PsA health outcome measures in clinical trials, only a few are used in CEAs. The methods by which utilities are estimated from these measures vary across cost-effectiveness studies. These differences should be considered when conducting cost-effectiveness research in psoriasis and PsA.