Fidelity to Program Specification of the National Health Service Digital Diabetes Prevention Program Behavior Change Technique Content and Underpinning Theory: Document Analysis

Improving retrospective intervention descriptions: Lessons learned from research on type 2 diabetes programmes in the United Kingdom and the Republic of Ireland

In recent years, multiple countries worldwide have implemented behavioural interventions within national healthcare systems. Describing the content of these interventions is critical to improve their implementation, replication, and effectiveness, as well as to advance behavioural science. Tools, such as the Behaviour Change Technique Taxonomy, can enhance the quality of intervention description and reporting. As interventions are frequently developed without the use of such tools, retrospective coding of existing interventions to accurately characterise their content is becoming more common. However, the use of these tools for retrospective coding poses various challenges, the discussion of which has been neglected to date. This commentary discusses the challenges encountered when retrospectively describing the content of five nationally implemented programmes for type 2 diabetes in the United Kingdom and the Republic of Ireland and suggests recommendations to tackle these challenges. We present important methodological, practical, and ethical considerations for researchers to reflect on, relevant to the retrospective description of existing interventions. Specifically, we discuss (i) the importance of positive relationships and collaboration with intervention stakeholders, (ii) the practical and ethical considerations when analysing the content of implemented interventions, (iii) the independence of research teams and the potential for misclassification of intervention content, and (iv) the challenges associated with the analysis of intervention content using behavioural science tools. There is a growing demand for more robust approaches to address the methodological, practical, and ethical challenges associated with such studies. The present commentary describes key issues to be considered by research teams, as well as concrete recommendations to improve the retrospective characterisation of intervention content.

Diabetes prevention at scale: Narrative review of findings and lessons from the DIPLOMA evaluation of the NHS Diabetes Prevention Programme in England

AIMS

The NHS Diabetes Prevention Programme (NHS DPP) is a large-scale, England-wide behaviour change programme for people at high risk of progressing to type 2 diabetes. We summarise the findings of our six-year DIPLOMA evaluation of its implementation and impact and highlight insights for future programmes.

METHODS

Using qualitative interviews, document analysis, observation, surveys and large dataset analysis, eight interlinked work packages considered: equity of access; implementation; service delivery and fidelity; programme outcomes; comparative effectiveness and cost-effectiveness in reducing diabetes incidence; and patient decision making and experience.

RESULTS

Delivery of the NHS DPP encountered barriers across many aspects of the programme, and we identified inequalities in terms of the areas, organisations and patient populations most likely to engage with the programme. There was some loss of fidelity at all stages from commissioning to participant understanding. Despite these challenges, there was evidence of significant reductions in diabetes incidence at individual and population levels. The programme was cost-effective even within a short time period.

CONCLUSIONS

Despite the challenge of translating research evidence into routine NHS delivery at scale, our findings suggest that an individual-level approach to the prevention of type 2 diabetes in a 'high-risk' population was more effective than usual care. By embedding evaluation with programme delivery and working closely with the NHS DPP team, we provided actionable insights for improving communications with potential participants, supporting primary care referral, honing the delivery model with better provider relationships and more patient choice, increasing understanding of behaviour change techniques, and enriching the educational and health coaching content.

Engagement with a nationally-implemented digital behaviour change intervention: Usage patterns over the 9-month duration of the National Health Service Digital Diabetes Prevention Programme

Background

Digital behaviour change interventions may offer a scalable way to promote weight loss by increasing physical activity and improving diet. However, user engagement is necessary for such benefits to be achieved. There is a dearth of research that assesses engagement with nationally implemented digital programmes offered in routine practice. The National Health Service Digital Diabetes Prevention Programme (NHS-DDPP) is a nine-month digital behaviour change intervention delivered by independent providers for adults in England who are at high risk of developing type 2 diabetes. This study reports engagement with the NHS-DDPP for users enrolled onto the programme over the nine-month duration.

Methods

Anonymous usage data was obtained for a cohort of service users (n = 1826) enrolled on the NHS-DDPP with three independent providers, between December 2020 and June 2021. Usage data were obtained for time spent in app, and frequency of use of NHS-DDPP intervention features in the apps including self-monitoring, goal setting, receiving educational content (via articles) and social support (via health coaches and group forums), to allow patterns of usage of these key features to be quantified across the nine-month intervention. Median usage was calculated within nine 30-day engagement periods to allow a longitudinal analysis of the dose of usage for each feature.

Results

App usage declined from a median of 32 min (IQR 191) in month one to 0 min (IQR 14) in month nine. Users self-monitored their behaviours (e.g., physical activity and diet) a median of 117 times (IQR 451) in the apps over the nine-month programme. The open group discussion forums were utilised less regularly (accessed a median of 0 times at all time-points). There was higher engagement with some intervention features (e.g., goal setting) when support from a health coach was linked to those features.

Conclusions

App usage decreased over the nine-month programme, although the rate at which the decrease occurred varied substantially between individuals and providers. Health coach support may promote engagement with specific intervention features. Future research should assess whether engagement with particular features of digital diabetes prevention programmes is associated with outcomes such as reduced bodyweight and HbA1c levels.

Design and Early Use of the Nationally Implemented Healthier You National Health Service Digital Diabetes Prevention Programme: Mixed Methods Study

BACKGROUND

The Healthier You National Health Service Digital Diabetes Prevention Programme (NHS-digital-DPP) is a 9-month digital behavior change intervention delivered by 4 independent providers that is implemented nationally across England. No studies have explored the design features included by service providers of digital diabetes prevention programs to promote engagement, and little is known about how participants of nationally implemented digital diabetes prevention programs such as this one make use of them.

OBJECTIVE

This study aimed to understand engagement with the NHS-digital-DPP. The specific objectives were to describe how engagement with the NHS-digital-DPP is promoted via design features and strategies and describe participants' early engagement with the NHS-digital-DPP apps.

METHODS

Mixed methods were used. The qualitative study was a secondary analysis of documents detailing the NHS-digital-DPP intervention design and interviews with program developers (n=6). Data were deductively coded according to an established framework of engagement with digital health interventions. For the quantitative study, anonymous use data collected over 9 months for each provider representing participants' first 30 days of use of the apps were obtained for participants enrolled in the NHS-digital-DPP. Use data fields were categorized into 4 intervention features (Track, Learn, Coach Interactions, and Peer Support). The amount of engagement with the intervention features was calculated for the entire cohort, and the differences between providers were explored statistically.

RESULTS

Data were available for 12,857 participants who enrolled in the NHS-digital-DPP during the data collection phase. Overall, 94.37% (12,133/12,857) of those enrolled engaged with the apps in the first 30 days. The median (IQR) number of days of use was 11 (2-25). Track features were engaged with the most (number of tracking events: median 46, IQR 3-22), and Peer Support features were the least engaged with, a median value of 0 (IQR 0-0). Differences in engagement with features were observed across providers. Qualitative findings offer explanations for the variations, including suggesting the importance of health coaches, reminders, and regular content updates to facilitate early engagement.

CONCLUSIONS

Almost all participants in the NHS-digital-DPP started using the apps. Differences across providers identified by the mixed methods analysis provide the opportunity to identify features that are important for engagement with digital health interventions and could inform the design of other digital behavior change interventions.

Description of the nationally implemented National Health Service digital diabetes prevention programme and rationale for its development: mixed methods study

BACKGROUND

The National Health Service (NHS) Digital Diabetes Prevention Programme (DDPP) is a behaviour change programme for adults in England who are at high risk of developing type 2 diabetes. Four independent providers deliver the NHS-DDPP following a competitive tendering process. Although providers work to a single service specification, there is potential for some variation in the service across providers. This study (1) assesses fidelity of the structural features of the design of the NHS-DDPP compared to the service specification, (2) describes the structural features of delivery of the NHS-DDPP as implemented (3) reports developers' views on how the structural components of the NHS-DDPP were developed and why changes were made following implementation.

METHODS

Using mixed methods, we conducted a document review of providers' NHS-DDPP design and delivery documentation, and extracted information using the Template for Intervention Description and Replication checklist, which was adapted to capture features of digital delivery. Documentation was supplemented by content analysis of interviews with 12 health coaches involved in delivering the NHS-DDPP. Semi-structured interviews were also conducted with 6 programme developers employed by the digital providers.

RESULTS

Provider plans for the NHS-DDPP show relatively high fidelity to the NHS service specification. Despite this, there was wide variation in structural features of delivery of the NHS-DDPP across providers, particularly for delivery of 'support' (e.g. use, dose and scheduling of health coaching and/or group support). Interviews with developers of the programmes showed that much of this variation is likely to be attributable to the origin of each provider's programme, which was usually a pre-existing programme that was adapted to conform to the NHS-DDPP service specification. The NHS-DDPP is continually improved and developed based on user experience feedback and research conducted by the providers.

CONCLUSION

Indirect evidence suggests that variation in delivery of support could affect effectiveness of the NHS-DDPP. A priority for future research is ascertaining whether the variation in delivery of the NHS-DDPP across providers is related to any differences in health outcomes. It is recommended that future rounds of commissioning the NHS-DDPP pre-specify the type of support participants should receive, including expected dose and scheduling.

Changes in a Digital Type 2 Diabetes Self-management Intervention During National Rollout: Mixed Methods Study of Fidelity

BACKGROUND

"Healthy Living for People with type 2 Diabetes (HeLP-Diabetes)" was a theory-based digital self-management intervention for people with type 2 diabetes mellitus that encouraged behavior change using behavior change techniques (BCTs) and promoted self-management. HeLP-Diabetes was effective in reducing HbA1c levels in a randomized controlled trial (RCT). National Health Service (NHS) England commissioned a national rollout of HeLP-Diabetes in routine care (now called "Healthy Living"). Healthy Living presents a unique opportunity to examine the fidelity of the national rollout of an intervention originally tested in an RCT.

OBJECTIVE

This research aimed to describe the Healthy Living BCT and self-management content and features of intervention delivery, compare the fidelity of Healthy Living with the original HeLP-Diabetes intervention, and explain the reasons for any fidelity drift during national rollout through qualitative interviews.

METHODS

Content analysis of Healthy Living was conducted using 3 coding frameworks (objective 1): the BCT Taxonomy v1, a new coding framework for assessing self-management tasks, and the Template for Intervention Description and Replication. The extent to which BCTs and self-management tasks were included in Healthy Living was compared with published descriptions of HeLP-Diabetes (objective 2). Semistructured interviews were conducted with 9 stakeholders involved in the development of HeLP-Diabetes or Healthy Living to understand the reasons for any changes during national rollout (objective 3). Qualitative data were thematically analyzed using a modified framework approach.

RESULTS

The content analysis identified 43 BCTs in Healthy Living. Healthy Living included all but one of the self-regulatory BCTs ("commitment") in the original HeLP-Diabetes intervention. Healthy Living was found to address all areas of self-management (medical, emotional, and role) in line with the original HeLP-Diabetes intervention. However, 2 important changes were identified. First, facilitated access by a health care professional was not implemented; interviews revealed this was because general practices had fewer resources in comparison with the RCT. Second, Healthy Living included an additional structured web-based learning curriculum that was developed by the HeLP-Diabetes team but was not included in the original RCT; interviews revealed that this was because of changes in NHS policy that encouraged referral to structured education. Interviewees described how the service provider had to reformat the content of the original HeLP-Diabetes website to make it more usable and accessible to meet the multiple digital standards required for implementation in the NHS.

CONCLUSIONS

The national rollout of Healthy Living had good fidelity to the BCT and self-management content of HeLP-Diabetes. Important changes were attributable to the challenges of scaling up a digital intervention from an RCT to a nationally implemented intervention, mainly because of fewer resources available in practice and the length of time since the RCT. This study highlights the importance of considering implementation throughout all phases of intervention development.