All interventions are complex, but some are more complex than others: using iCAT_SR to assess complexity

Clinical validation of grouping conservative treatments in neck pain for use in a network meta-analysis: a Delphi consensus study

BACKGROUND

A network meta-analysis aims to help clinicians make clinical decisions on the most effective treatment for a certain condition. Neck pain is multifactorial, with various classification systems and treatment options. Classifying patients and grouping interventions in clinically relevant treatment nodes for a NMA is essential, but this process is poorly defined.

OBJECTIVE

Our aim is to obtain consensus among experts on neck pain classifications and the grouping of interventions into nodes for a future network meta-analysis.

DESIGN

A Delphi consensus study involving neck pain experts worldwide.

METHODS

We invited authors of neck pain clinical practice guidelines published from 2014 onwards. The Delphi baseline questionnaire was developed based on the findings of a scoping review, including four items on classifications and 19 nodes. Participants were asked to record their level of agreement on a seven-point Likert scale or using Yes/No/Not sure answer options for the various statements. We used descriptive analysis to summarise the responses on each statement with content analysis of the free-text comments.

RESULTS

In total, 18/80 experts (22.5%) agreed to participate in one or more Delphi rounds. We needed three rounds to reach consensus for two classification of neck pain: one based on aetiology and one on duration. In addition, we also reached consensus on the grouping of interventions, including a definition of each node, with the number of nodes reduced to 17.

CONCLUSION

With this consensus we clinically validated two neck pain classifications and grouped conservative treatments into 17 well-defined and clinically relevant nodes.

Systematic review of process evaluations of interventions in trials investigating sedentary behaviour in adults

OBJECTIVES

To systematically review and synthesise findings from process evaluations of interventions in trials which measured sedentary behaviour as an outcome in adults to explore: (1) how intervention content, implementation, mechanisms of impact and context influence outcomes and (2) how these interventions are experienced from different perspectives (participants, carers, staff).

DESIGN

Systematic review and narrative synthesis underpinned by the Medical Research Council process evaluation framework.

DATA SOURCES

Databases searches were conducted in March 2019 then updated in May 2020 and October 2021 in: CINAHL, SPORTDiscus, Cochrane Database of Systematic Reviews, Cochrane Central Register of Controlled Trials, AMED; EMBASE, PsycINFO, MEDLINE, Web of Science and ProQuest Dissertations & Theses.

ELIGIBILITY CRITERIA

We included: Process evaluations of trials including interventions where sedentary behaviour was measured as an outcome in adults aged 16 or over from clinical or non-clinical populations. We excluded studies if interventions were delivered in educational or workplace settings, or if they were laboratory studies focused on immediate effects of breaking sitting.

DATA EXTRACTION AND SYNTHESIS

Two independent reviewers extracted and coded data into a framework and assessed the quality of studies using the Mixed Methods Appraisal Tool. We synthesised findings using a narrative approach.

RESULTS

17 process evaluations were included. Five interventions focused on reducing sedentary behaviour or sitting time, 12 aimed to increase physical activity or promote healthier lifestyles. Process evaluations indicated changes in sedentary behaviour outcomes were shaped by numerous factors including: barriers (eg, staffing difficulties and scheduling problems) and facilitators (eg, allowing for flexibility) to intervention delivery; contextual factors (eg, usual lifestyle and religious events) and individual factors (eg, pain, tiredness, illness, age and individual preferences).

DISCUSSION

Intervention requires careful consideration of different factors that could influence changes in sedentary behaviour outcomes to ensure that interventions can be tailored to suit different individuals and groups.

PROSPERO REGISTRATION NUMBER

CRD42018087403.

How has implementation been incorporated in health technology assessments in the United Kingdom? A systematic rapid review

OBJECTIVES

Health interventions in a clinical setting may be complex. This is particularly true of clinical interventions which require systems reorganization or behavioural change, and/or when implementation involves additional challenges not captured within a clinical trial setting. Medical Research Council guidance on complex interventions highlights the need to consider economic evaluation alongside implementation. However, the extent to which this guidance has been adhered to, and how, is unclear. The failure to incorporate implementation within the evaluation of an intervention may hinder the translation of research findings into routine practice. This will have consequences for patient care. This study examined the methods used to address implementation within health research conducted through funding from the National Institute for Health Research (NIHR) Health Technology Assessment (HTA) programme.

METHODS

We conducted a rapid review using a systematic approach. We included all NIHR HTA monographs which contained the word "implementation" within the title or abstract published between 2014 and 2020. We assessed the studies according to existing recommendations for specifying and reporting implementation approaches in research. Additional themes which were not included in the recommendation, but were of particular relevance to our research question, were also identified and summarized in a narrative synthesis.

RESULTS

The extent to which implementation was formally incorporated, and defined, varied among studies. Methods for examining implementation ranged from single stakeholder engagement events to the more comprehensive process evaluation. There was no obvious pattern as to whether approaches to implementation had evolved over recent years. Approximately 50% (22/42) of studies included an economic evaluation. Of these, two studies included the use of qualitative data obtained within the study to quantitatively inform aspects relating to implementation and economic evaluation in their study.

DISCUSSION

A variety of approaches were identified for incorporating implementation within an HTA. However, they did not go far enough in terms of incorporating implementation into the actual design and evaluation. To ensure the implementation of clinically effective and cost-effective interventions, we propose that further guidance on how to incorporate implementation within complex interventions is required. Incorporating implementation into economic evaluation provides a step in this direction.

LIVE@Home.Path-innovating the clinical pathway for home-dwelling people with dementia and their caregivers: study protocol for a mixed-method, stepped-wedge, randomized controlled trial

BACKGROUND

The global health challenge of dementia is exceptional in size, cost and impact. It is the only top ten cause of death that cannot be prevented, cured or substantially slowed, leaving disease management, caregiver support and service innovation as the main targets for reduction of disease burden. Institutionalization of persons with dementia is common in western countries, despite patients preferring to live longer at home, supported by caregivers. Such complex health challenges warrant multicomponent interventions thoroughly implemented in daily clinical practice. This article describes the rationale, development, feasibility testing and implementation process of the LIVE@Home.Path trial.

METHODS

The LIVE@Home.Path trial is a 2-year, multicenter, mixed-method, stepped-wedge randomized controlled trial, aiming to include 315 dyads of home-dwelling people with dementia and their caregivers, recruited from 3 municipalities in Norway. The stepped-wedge randomization implies that all dyads receive the intervention, but the timing is determined by randomization. The control group constitutes the dyads waiting for the intervention. The multicomponent intervention was developed in collaboration with user-representatives, researchers and stakeholders to meet the requirements from the national Dementia Plan 2020. During the 6-month intervention period, the participants will be allocated to a municipal coordinator, the core feature of the intervention, responsible for regular contact with the dyads to facilitate L: Learning, I: Innovation, V: Volunteering and E: Empowerment (LIVE). The primary outcome is resource utilization. This is measured by the Resource Utilization in Dementia (RUD) instrument and the Relative Stress Scale (RSS), reflecting that resource utilization is more than the actual time required for caring but also how burdensome the task is experienced by the caregiver.

DISCUSSION

We expect the implementation of LIVE to lead to a pathway for dementia treatment and care which is cost-effective, compared to treatment as usual, and will support high-quality independent living, at home.

TRIAL REGISTRATION

ClinicalTrials.gov: NCT04043364. Registered on 15 March 2019.

Application of the intervention Complexity Assessment Tool for Systematic Reviews within a Cochrane review: an illustrative case study

Background: The intervention Complexity Assessment Tool for Systematic Reviews (iCAT_SR) has been developed to facilitate detailed assessments of intervention complexity in systematic reviews. Worked examples of the tool's application are needed to promote its use and refinement. The aim of this case study was to apply the iCAT_SR to a subset of 20 studies included in a Cochrane review of interventions aimed at improving appropriate polypharmacy in older people. Methods: Interventions were assessed independently by two authors using the six core iCAT_SR dimensions: (1) 'Target organisational levels/categories'; (2) 'Target behaviour/actions'; (3) 'Active intervention components'; (4) 'Degree of tailoring'; (5) 'Level of skill required by intervention deliverers'; (6) 'Level of skill required by intervention recipients'. Attempts were made to apply four optional dimensions: 'Interaction between intervention components'; 'Context/setting'; 'Recipient/provider factors'; 'Nature of causal pathway'. Inter-rater reliability was assessed using Cohen's Kappa coefficient. Disagreements were resolved by consensus discussion. The findings are presented narratively. Results: Assessments involving the core iCAT_SR dimensions showed limited consistency in intervention complexity across included studies, even when categorised according to clinical setting. Interventions were delivered across various organisational levels and categories (i.e. healthcare professionals and patients) and typically comprised multiple components. Intermediate skill levels were required by those delivering and receiving the interventions across all studies. A lack of detail in study reports precluded application of the iCAT_SR's optional dimensions. The inter-rater reliability was substantial (Cohen's Kappa = 0.75) Conclusions: This study describes the application of the iCAT_SR to studies included in a Cochrane systematic review. Future intervention studies need to ensure more detailed reporting of interventions, context and the causal pathways underlying intervention effects to allow a more holistic understanding of intervention complexity and facilitate replication in other settings. The experience gained has helped to refine the original guidance document relating to the application of iCAT_SR.

Evaluation and facilitation of intervention fidelity in community exercise programs through an adaptation of the TIDier framework

Background

Despite high quality evidence supporting multiple physical and cognitive benefits of community-based exercise for people after stroke, there is little understanding on how to facilitate uptake of these research findings to real-world programs. A common barrier is a lack of standardised training for community fitness instructors, which hampers the ability to train more instructors to deliver the program as it was designed. Scaling up program delivery, while maintaining program fidelity, is complex. The objective of this research is to explore novel use of the Template for Intervention Description and Replication (TIDier) framework to evaluate and support implementation fidelity of a community exercise program.

Methods

We embedded intervention fidelity evaluation into an inaugural training program for fitness instructors who were to deliver the Fitness and Mobility Exercise Program for stroke, which has established efficacy. The training program consisted of a face-to-face workshop followed by 3 worksite ‘audit and feedback coaching cycles’ provided over 3 iterations of the 12-week program offered over 1 year. A modified TIDIER checklist (with 2 additional criteria) was used within the training workshop to clarify the key ‘active ingredients’ that were required for program fidelity, and secondly as a basis for the audit and feedback process enabling the quantitative measurement of fidelity. Data were collected from audits of observed classes and from a survey provided by fitness instructors who implemented the program.

Results

We demonstrated the feasibility of the TIDier checklist to capture 14 essential items for implementation evaluation of a complex exercise intervention for people with chronic health conditions over 3 iterations of the program. Based on the audit tool, program fidelity was high and improved over time. Three content areas for workplace coaching (intensity monitoring, space, and educational tips) were identified from the audit tool and were addressed.

Conclusion

Training of staff to deliver exercises to high need populations utilising workshops and workplace coaching that used the TIDier framework for training, onsite audit and feedback resulted in a high level of fidelity to the program principles. A novel checklist based on the TIDier framework was useful for embedding implementation fidelity in complex community-based interventions.

Taking account of context in systematic reviews and guidelines considering a complexity perspective

Systematic review teams and guideline development groups face considerable challenges when considering context within the evidence production process. Many complex interventions are context-dependent and are frequently evaluated within considerable contextual variation and change. This paper considers the extent to which current tools used within systematic reviews and guideline development are suitable in meeting these challenges. The paper briefly reviews strengths and weaknesses of existing approaches to specifying context. Illustrative tools are mapped to corresponding stages of the systematic review process. Collectively, systematic review and guideline production reveals a rich diversity of frameworks and tools for handling context. However, current approaches address only specific elements of context, are derived from primary studies which lack information or have not been tested within systematic reviews. A hypothetical example is used to illustrate how context could be integrated throughout the guideline development process. Guideline developers and evidence synthesis organisations should select an appropriate level of contextual detail for their specific guideline that is parsimonious and yet sensitive to health systems contexts and the values, preferences and needs of their target populations.