Evaluating stakeholder involvement in building a decision support tool for NHS health checks: co-producing the WorkHORSE study

A Value Framework to Assess Patient-Facing Digital Health Technologies That Aim to Improve Chronic Disease Management: A Delphi Approach

OBJECTIVES

Digital health technologies (DHTs) can optimise healthcare costs and improve quality and efficiency of care. However, the fast-paced rate of innovation and varying evidence standards can make it difficult for decision-makers to assess these technologies in an efficient and evidence-based manner. We sought to develop a comprehensive framework to assess the value of novel patient-facing DHTs used to manage chronic diseases by eliciting stakeholder value preferences.

METHODS

Literature review and primary data collection from a three-round web-Delphi exercise was utilized. 79 participants from 5 stakeholder groups (patients, physicians, industry, decision makers, and influencers) and 3 countries (United States of America, United Kingdom, and Germany) took part. Likert scale data were statistically analyzed to determine intergroup differences in both country and stakeholder groups, stability of results, and overall consensus.

RESULTS

The resulting co-created framework comprised 33 stable indicators with consensus from quantitative value judgments across domains: health inequalities, data rights and governance, technical and security, economic characteristics, clinical characteristics, and user preferences. Lack of stakeholder consensus was observed on the importance of value-based care models, optimizing resources for sustainable systems, and stakeholder involvement in DHT design, development, and implementation; however, this was because of high rates of neutrality and not negative judgments. Supply-side actors and academic experts were the most unstable stakeholder groups.

CONCLUSION

Stakeholder value judgments revealed a need for a coordinated regulatory and health technology assessment policy response that updates laws to meet technological innovations, offers a pragmatic approach to evidence standards to assess DHTs, and involves stakeholders to understand and meet their needs.

Four Aspects Affecting Health Economic Decision Models and Their Validation

Health care decision makers in many jurisdictions use cost-effectiveness analysis based on health economic decision models for policy decisions regarding coverage and price negotiation for medicines and medical devices. While validation of health economic decision models has always been considered important, many reviews of model-based cost-effectiveness studies report limitations regarding their validation. The current opinion paper discusses four aspects of current health economic decision modeling with relevance for future directions in model validation: increased use of complex models, international cooperation, open-source modeling, and stakeholder involvement. First, new, more complex clinical study designs and treatment strategies may require relatively complex model structures and/or input data analyses. Simultaneously, more widespread technical knowledge along with wider data availability have led to a broader range of model types. This puts extra requirements on model validation and transparency. Second, increased international cooperation of policy makers and, in particular, health technology assessment (HTA) authorities in performing model assessments is discussed in relation to the repeated use of health economic models (multi-use disease models). We argue such coordinated efforts may benefit model validity. Third, open-source modeling is discussed as one possible answer to increased transparency requirements. Finally, involvement of all relevant stakeholders throughout the whole decision process is an ongoing development that necessarily also includes health economic modeling. We argue this implies that model validity should be considered in a broader perspective, with more focus on conceptual modeling, model transparency, accuracy requirements, and choice of relevant model outcomes than previously.

Improving peripheral venous cannula insertion in children: a mixed methods study to develop the DIVA key

OBJECTIVE

To develop and validate a difficult intravenous access risk assessment and escalation pathway, to increase first time intravenous insertion success in paediatrics.

METHODS

Mixed methods underpinned by literature and co-production principles. Iterative development of the instrument was informed through semi-structured interviews and stakeholder workshops. The instrument includes a risk assessment, inserter skill self-assessment, and escalation pathways. Reproducibility, reliability, and acceptability were evaluated in a prospective cohort study at a quaternary paediatric hospital in Australia.

RESULTS

Interview data (three parents, nine clinicians) uncovered two themes: i) Recognition of children with DIVA and subsequent escalation is ad hoc and problematic; and ii) Resources and training impact inserter confidence and ability. Three workshops were delivered at monthly intervals (February-April 2020) involving 21 stakeholders culminating in the co-production of the "DIVA Key". The DIVA Key was evaluated between May-December 2020 in 78 children; 156 clinicians. Seventy-eight paired assessments were undertaken with substantial agreement (concordance range = 81.5 to 83.0%) between the assessors. Interrater reliability of the DIVA risk assessment was moderate (kappa = 0.71, 95% CI 0.63-0.80). The DIVA Key predicted multiple insertion attempts for red (high risk) DIVA classification (relative risk ratio 5.7, 95% CI 1.2-27.1; reference low risk). Consumer and clinician satisfaction with DIVA Key was high (median (IQR) = 10 [8-10]; 8 [8-10 respectively).

CONCLUSION

The DIVA Key is a straightforward, reliable instrument with inbuilt escalation pathway to support the identification of children with difficult intravenous access.

Modelling tool to support decision-making in the NHS Health Check programme: workshops, systematic review and co-production with users

BACKGROUND

Local authorities in England commission the NHS Health Check programme to invite everyone aged 40-74 years without pre-existing conditions for risk assessment and eventual intervention, if needed. However, the programme's effectiveness, cost-effectiveness and equity impact remain uncertain.

AIM

To develop a validated open-access flexible web-based model that enables local commissioners to quantify the cost-effectiveness and potential for equitable population health gain of the NHS Health Check programme.

OBJECTIVES

The objectives were as follows: (1) co-produce with stakeholders the desirable features of the user-friendly model; (2) update the evidence base to support model and scenario development; (3) further develop our computational model to allow for developments and changes to the NHS Health Check programme and the diseases it addresses; (4) assess the effectiveness, cost-effectiveness and equity of alternative strategies for implementation to illustrate the use of the tool; and (5) propose a sustainability and implementation plan to deploy our user-friendly computational model at the local level.

DESIGN

Co-production workshops surveying the best-performing local authorities and a systematic literature review of strategies to increase uptake of screening programmes informed model use and development. We then co-produced the workHORSE (working Health Outcomes Research Simulation Environment) model to estimate the health, economic and equity impact of different NHS Health Check programme implementations, using illustrative-use cases.

SETTING

Local authorities in England.

PARTICIPANTS

Stakeholders from local authorities, Public Health England, the NHS, the British Heart Foundation, academia and other organisations participated in the workshops. For the local authorities survey, we invited 16 of the best-performing local authorities in England.

INTERVENTIONS

The user interface allows users to vary key parameters that represent programme activities (i.e. invitation, uptake, prescriptions and referrals). Scenarios can be compared with each other.

MAIN OUTCOME MEASURES

Disease cases and case-years prevented or postponed, incremental cost-effectiveness ratios, net monetary benefit and change in slope index of inequality.

RESULTS

The survey of best-performing local authorities revealed a diversity of effective approaches to maximise the coverage and uptake of NHS Health Check programme, with no distinct 'best buy'. The umbrella literature review identified a range of effective single interventions. However, these generally need to be combined to maximally improve uptake and health gains. A validated dynamic, stochastic microsimulation model, built on robust epidemiology, enabled service options analysis. Analyses of three contrasting illustrative cases estimated the health, economic and equity impact of optimising the Health Checks, and the added value of obtaining detailed local data. Optimising the programme in Liverpool can become cost-effective and equitable, but simply changing the invitation method will require other programme changes to improve its performance. Detailed data inputs can benefit local analysis.

LIMITATIONS

Although the approach is extremely flexible, it is complex and requires substantial amounts of data, alongside expertise to both maintain and run.

CONCLUSIONS

Our project showed that the workHORSE model could be used to estimate the health, economic and equity impact comprehensively at local authority level. It has the potential for further development as a commissioning tool and to stimulate broader discussions on the role of these tools in real-world decision-making.

FUTURE WORK

Future work should focus on improving user interactions with the model, modelling simulation standards, and adapting workHORSE for evaluation, design and implementation support.

STUDY REGISTRATION

This study is registered as PROSPERO CRD42019132087.

FUNDING

This project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 25, No. 35. See the NIHR Journals Library website for further project information.