Complex interventions: how "out of control" can a randomised controlled trial be?

Understanding the challenges and successes of implementing 'hybrid' interventions in healthcare settings: findings from a process evaluation of a patient involvement trial

INTRODUCTION

'Hybrid' interventions in which some intervention components are fixed across sites and others are flexible (locally created) are thought to allow for adaptation to the local context while maintaining fidelity. However, there is little evidence regarding the challenges and facilitators of implementing hybrid interventions. This paper reports on a process evaluation of a patient safety hybrid intervention called Your Care Needs You (YCNY). YCNY was tested in the Partners at Care Transitions (PACT) randomised controlled trial and aimed to enhance older patients and their families' involvement in their care in order to achieve safer transitions from hospital to home.

METHODS

The process evaluation took place across eight intervention wards taking part in the PACT trial. 23 interviews and 37 informal conversations were conducted with National Health Service (NHS) staff. Patients (n=19) were interviewed twice, once in hospital and once after discharge. Interviews with staff and patients concerned the delivery and experiences of YCNY. Ethnographic observations (n=81 hours) of relevant activities (eg, multidisciplinary team meetings, handovers, etc) were undertaken.

RESULTS

The main finding relates to how staff understood and engaged with YCNY, which then had a major influence on its implementation. While staff broadly valued the aims of YCNY, staff from seven out of the eight wards taking part in the process evaluation enacted YCNY in a mostly task-based manner. YCNY implementation often became a hurried activity which concentrated on delivering fixed intervention components rather than a catalyst for culture change around patient involvement. Factors such as understaffing, constraints on staff time and the COVID-19 pandemic contributed towards a 'taskification' of intervention delivery, which meant staff often did not have capacity to creatively devise flexible intervention components. However, one ward with a sense of distributed ownership of YCNY had considerable success implementing flexible components.

DISCUSSION

Hybrid interventions may allow aspects of an intervention to be adapted to the local context. However, the current constrained and pressured environment of the NHS left staff with little ability to creatively engage with devising flexible intervention components, despite recognising the need for and being motivated to deliver the intervention.

Hospital design for inpatient psychiatry: A realistic umbrella review

The evaluation of the effects of architecture and design in psychiatric hospitals primarily focuses on final outcomes, such as disease progression, and is made from the perspective of evidence-based medicine. Meanwhile, the evidence-informed, realist approach addresses how the intervention works. Understanding the underlying action mechanisms of the intervention is needed to facilitate its scaling-up and adaptation in new environments. This umbrella review reports in which ways architecture and design have an effect on patients' and staff experience in inpatient psychiatric hospital. The search was constructed around three key concepts (psychiatric hospital, design, and staff and patient outcomes) and was conducted across three reference databases (Embase, Medline, and PsychINFO). Academic and gray literature was analyzed. Information on design and architectural features in psychiatric hospitals, their effects on patients and staff experience, and the acting mechanisms enabling these effects were extracted. From 951 original references, 14 full texts were included in the analysis. Design and architectural features (e.g., aesthetic appeal of places, home-like environment) in psychiatric hospitals address patients' stress, boost social interaction, foster patients' autonomy and feelings of control, ensure respect for patient's privacy and dignity, and prevent under-and overstimulation. Using theory-driven evaluation may facilitate future hospital renovation and the evaluation of its effect.

The Secret Sauce? Taking the Mystery Out of Scaling-Up School-Based Physical Activity Interventions

Over the last 4 decades, physical activity researchers have invested heavily in determining "what works" to promote healthy behaviors in schools. Single and multicomponent school-based interventions that target physical education, active transportation, and/or classroom activity breaks effectively increased physical activity among children and youth. Yet, few of these interventions are ever scaled-up and implemented under real-world conditions and in diverse populations. To achieve population-level health benefits, there is a need to design school-based health-promoting interventions for scalability and to consider key aspects of the scale-up process. In this opinion piece, we aim to identify challenges and advance knowledge and action toward scaling-up school-based physical activity interventions. We highlight the key roles of planning for scale-up at the outset, scale-up pathways, trust among partners and program support, program adaptation, evaluation of scale-up, and barriers and facilitators to scaling-up. We draw upon our experience scaling-up effective school-based interventions and provide a solid foundation from which others can work toward bridging the implementation-to-scale-up gap.

Implementation of a fluid balance control strategy in critically ill patients: POINCARE-2 trial process evaluation

BACKGROUND

POINCARE-2 trial aimed to assess the effectiveness of a strategy designed to tackle fluid overload through daily weighing and subsequent administration of treatments in critically ill patients. Even in highly standardized care settings, such as intensive care units, effectiveness of such a complex intervention depends on its actual efficacy but also on the extent of its implementation. Using a process evaluation, we aimed to provide understanding of the implementation, context, and mechanisms of change of POINCARE-2 strategy during the trial, to gain insight on its effectiveness and inform the decision regarding the dissemination of the intervention.

METHODS

We conducted a mixed-method process evaluation following the Medical Research Council guideline. Both quantitative data derived from the trial, and qualitative data from semi-structured interviews with professionals were used to explain implementation, mechanisms of change of the POINCARE-2 strategy, as well as contextual factors potentially influencing implementation of the strategy.

RESULTS

Score of actual exposure to the strategy ranged from 29.1 to 68.2% during the control period, and from 61.9 to 92.3% during the intervention period, suggesting both potential contamination and suboptimal fidelity to the strategy. Lack of appropriate weighing devices, lack of human resources dedicated to research, pre-trial rooted prescription habits, and anticipated knowledge of the strategy have been identified as the main barriers to optimal implementation of the strategy in the trial context.

CONCLUSIONS

Both contamination and suboptimal fidelity to POINCARE-2 strategy raised concerns about a potential bias towards the null of intention-to-treat (ITT) analyses. However, optimal fidelity seemed reachable. Consequently, a clinical strategy should not be rejected solely on the basis of the negativity of ITT analyses' results. Our findings showed that, even in highly standardized care conditions, the implementation of clinical strategies may be hindered by numerous contextual factors, which demonstrates the critical importance of assessing the viability of an intervention, prior to any evaluation of its effectiveness.

TRIAL REGISTRATION

Number NCT02765009.

Epidaure Market-Effectiveness and transferability of a school-based intervention to improve healthy and sustainable food choices by schoolchildren: Protocol of a randomized controlled trial and qualitative study

CONTEXT

At least 40% of cancers are linked to environmental or behavioral factors, and dietary behavior appears to be a major lever. Epidaure Market is a prevention initiative developed using a method for co-constructing health promotion initiatives and prevention programs that stratifies evidence from the scientific literature and combines it with experiential knowledge (DEVA, TPB, BCT). It promotes a sustainable diet (i.e., healthy, ecological and ethical nutrition) among 5th and 4th grade students during the crucial period of adolescence, when these behaviors are often far from the recommendations.

METHOD

The protocol implemented was carried out in 72 middle school classes in the Montpellier and Dijon academies. The intervention included teaching sessions and a virtual supermarket game, integrated into the school curriculum and delivered by science teachers. Effectiveness is tested in a cluster randomized controlled trial with 3 evaluation times (pre- and post-intervention and 1 follow-up). The study also includes an implementation assessment, with process analysis and implementation elements, as well as a transferability assessment based on key functions (FIC model and Astaire grid).

EXPECTED OUTCOMES

The study is still underway within the school. The primary expected outcome is a positive influence on the motives underlying food choices to move towards a sustainable diet. Secondary expectations involve changes in variables such as self-efficacy and perceived social norms, as well as an increase in knowledge about healthy eating. We also expect the qualitative approaches to provide information on the deployment process in the new territories.

DISCUSSION

The study aims not only to demonstrate the effectiveness of Epidaure Market, but also to identify the optimal conditions for its nationwide implementation in France's middle schools. Ultimately, the initiative aims to help reduce the incidence of cancer by promoting healthier eating habits among teenagers.

Similarities and Differences Between Pragmatic Trials and Hybrid Effectiveness-Implementation Trials

Pragmatism in clinical trials is focused on increasing the generalizability of research findings for routine clinical care settings. Hybridism in clinical trials (i.e., assessing both clinical effectiveness and implementation success) is focused on speeding up the process by which evidence-based practices are developed and adopted into routine clinical care. Even though pragmatic trial methodologies and implementation science evolved from very different disciplines, Pragmatic Trials and Hybrid Effectiveness-Implementation Trials share many similar design features. In fact, these types of trials can easily be conflated, creating the potential for investigators to mislabel their trial type or mistakenly use the wrong trial type to answer their research question. Blurred boundaries between trial types can hamper the evaluation of grant applications, the scientific interpretation of findings, and policy-making. Acknowledging that most trials are not pure Pragmatic Trials nor pure Hybrid Effectiveness-Implementation Trials, there are key differences in these trial types and they answer very different research questions. The purpose of this paper is to clarify the similarities and differences of these trial types for funders, researchers, and policy-makers. In addition, recommendations are offered to help investigators choose, label, and operationalize the most appropriate trial type to answer their research question. These recommendations complement existing reporting guidelines for clinical effectiveness trials (TIDieR) and implementation trials (StaRI).

Scaling-Out Digitally Enabled Integrated Care in Europe Through Good Practices Transfer: The JADECARE Study

Introduction

The absence of a coordinated approach to health and social care compromises the ability of health systems to provide universal, equitable, high-quality, and financially sustainable care. Transferring evidence-based practices focused on digitally-enabled integrated care to new contexts can overcome this challenge if implementation is satisfactory. This paper presents the scaling-out methodology that JADECARE has designed to spread effective innovative practices across Europe.

Methodology

The scaling-out methodology pretends to guide the Next Adopters in the transfer and adoption of practices, whereas increasing their implementation capacity and providing an evaluation framework to assess impact and success.

Discussion

JADECARE scaling-out effort is based on guiding principles found in the literature such as the balance between fidelity to the original practice and the degree of adaptation required to fit the new context, the need for capacity building in implementation to bridge the gap between research and routine practice and the focus on explaining why, for whom and in what circumstances an intervention works.

Conclusion

The JADECARE scaling-out methodology is theory-driven and pragmatic and aims to facilitate the transfer of complex interventions across different contexts.

Implementation of Complex Suicide Prevention Interventions: Insights into Barriers, Facilitators and Lessons Learned

INTRODUCTION

Effective suicide prevention interventions are infrequently translated into practice and policy. One way to bridge this gap is to understand the influence of theoretical determinants on intervention delivery, adoption, and sustainment and lessons learned. This study aimed to examine barriers, facilitators and lessons learned from implementing complex suicide prevention interventions across the world.

METHODS AND MATERIALS

This study was a secondary analysis of a systematic review of complex suicide prevention interventions, following updated PRISMA guidelines. English published records and grey literature between 1990 and 2022 were searched on PubMed, CINAHL, PsycINFO, ProQuest, SCOPUS and CENTRAL. Related reports were organized into clusters. Data was extracted from clusters of reports on interventions and were mapped using the updated Consolidated Framework for Implementation Research.

RESULTS

The most frequently-reported barriers were reported within the intervention setting and were related to the perceived appropriateness of interventions within settings; shared norms, beliefs; and maintaining formal and informal networks and connections. The most frequently reported facilitators concerned individuals' motivation, capability/capacity, and felt need. Lessons learned focused on the importance of tailoring the intervention, responding to contextual needs and the importance of community engagement throughout the process.

CONCLUSION

This study emphasizes the importance of documenting and analyzing important influences on implementation. The complex interplay between the contextual determinants and implementation is discussed. These findings contribute to a better understanding of barriers and facilitators salient for implementation of complex suicide prevention interventions.

A new framework for developing and evaluating complex interventions: update of Medical Research Council guidance

The UK Medical Research Council's widely used guidance for developing and evaluating complex interventions has been replaced by a new framework, commissioned jointly by the Medical Research Council and the National Institute for Health Research, which takes account of recent developments in theory and methods and the need to maximise the efficiency, use, and impact of research.

Process evaluation of a complex, multilevel, multicomponent scheme for the prevention and control of non-communicable diseases in Tamil Nadu, India: A mixed-methods protocol

Background

Non-communicable diseases (NCDs) are the leading cause of morbidity and mortality in India, necessitating development of multilevel and multicomponent interventions. Makkalai Thedi Maruthuvam (MTM) is a complex multilevel, multicomponent intervention developed and implemented by the south Indian State of Tamil Nadu. The scheme aims to deliver services for preventing and controlling diabetes, and hypertension at doorstep. This paper describes the protocol for planning and conducting the process evaluation of the MTM scheme.

Methods and analysis

The process evaluation uses mixed methods (secondary data analysis, key informant interviews, in-depth interviews, conceptual content analysis of documents, facility-based survey and non-participant observation) to evaluate the implementation of the MTM scheme. The broad evaluation questions addressed the fidelity, contexts, mechanisms of impact and challenges encountered by the scheme using the Consolidated Framework for Implementation Research (CFIR) framework. The specific evaluation questions addressed selected inputs and processes identified as critical to implementation by the stakeholders. The CFIR framework will guide the thematic analysis of the qualitative interviews to explore the adaptations and deviations introduced during implementation in various contexts. The quantitative data on the indicators developed for the specific evaluation questions will be cleaned and descriptively analysed.

PReventing and Approaching Crises for frail community-dwelling patients Through Innovative Care (PRACTIC): protocol for an effectiveness cluster randomised controlled trial

BACKGROUND

Demographic changes, with an increasing number and proportion of older people with multimorbidity and frailty, will put more pressure on home care services in municipalities. Frail multimorbid people receiving home care services are at high risk of developing crises, defined as critical challenges and symptoms, which demand immediate and new actions. The crises often result in adverse events, coercive measures, and acute institutionalisation. There is a lack of evidence-based interventions to prevent and resolve crises in community settings.

METHODS

This is a participatory action research design (PAR) in a 6-month cluster randomised controlled trial (RCT). The trial will be conducted in 30 municipalities, including 150 frail community-dwelling participants receiving home care services judged by the services to be at risk of developing crisis. Each municipality (cluster) will be randomised to receive either the locally adapted TIME intervention (the intervention group) or care as usual (the control group). The Targeted Interdisciplinary Model for Evaluation and Treatment of Neuropsychiatric Symptoms (TIME) is a manual-based, multicomponent programme that includes a rigorous assessment of the crisis, one or more interdisciplinary case conferences, and the testing and evaluation of customised treatment measures. PAR in combination with an RCT will enhance adaptations of the intervention to the local context and needs. The primary outcome is as follows: difference in change between the intervention and control groups in individual goal achievement to resolve or reduce the challenges of the crises between baseline and 3 months using the PRACTIC Goal Setting Interview (PGSI). Among the secondary outcomes are the difference in change in the PGSI scale at 6 months and in neuropsychiatric symptoms (NPSs), quality of life, distress perceived by professional carers and next of kin, and institutionalisation at 3 and 6 months.

DISCUSSION

Through customised interventions that involve patients, the next of kin, the social context, and health care services, crises may be prevented and resolved. The PReventing and Approaching Crises for frail community-dwelling patients Through Innovative Care (PRACTIC) study will enhance innovation for health professionals, management, and users in the development of new knowledge and a new adapted approach towards crises.

TRIAL REGISTRATION

ClinicalTrials.gov identifier: NCT05651659. Registered 15.12.22.

HCI Contributions in Mental Health: A Modular Framework to Guide Psychosocial Intervention Design

Many people prefer psychosocial interventions for mental health care or other concerns, but these interventions are often complex and unavailable in settings where people seek care. Intervention designers use technology to improve user experience or reach of interventions, and HCI researchers have made many contributions toward this goal. Both HCI and mental health researchers must navigate tensions between innovating on and adhering to the theories of change that guide intervention design. In this paper, we propose a framework that describes design briefs and evaluation approaches for HCI contributions at the scopes of capabilities, components, intervention systems, and intervention implementations. We show how theories of change (from mental health) can be translated into design briefs (in HCI), and that these translations can bridge and coordinate efforts across fields. It is our hope that this framework can support researchers in motivating, planning, conducting, and communicating work that advances psychosocial intervention design.

Adapting an Effective Health-Promoting Intervention-Choose to Move-for Chinese Older Adults in Canada

Evidence is sparse on how community-based health-promoting programs can be culturally adapted for racially minoritized, immigrant older adult populations. Choose to Move (CTM) is an evidence-based health-promoting program that enhances physical activity and mobility and diminished social isolation and loneliness in older adults in British Columbia, Canada. However, racially minoritized older adults were not reached in initial offerings. We purposively sampled CTM delivery staff (n = 8) from three not-for-profit organizations, in Metro Vancouver, British Columbia, that serve Chinese older adults. We used semistructured interviews, ethnographic observations, and meeting minutes to understand delivery staff's perspectives on factors that influence CTM adaptations for Chinese older adults. Deductive framework analysis guided by an adaptation framework, Framework for Reporting Adaptations and Modifications-Enhanced, found three dominant cultural- and immigration-related factors influenced CTM adaptations for Chinese older adults: (a) prioritizations, (b) familiarity, and (c) literacy. Findings may influence future program development and delivery to meet the needs of racially minoritized older adult populations.

Protocol for the process evaluation of an intervention to improve antenatal smoking cessation support (MOHMQuit) in maternity services in New South Wales, Australia

INTRODUCTION

Smoking cessation in pregnancy remains a public health priority. Our team used the Behaviour Change Wheel to develop the Midwives and Obstetricians Helping Mothers to Quit smoking (MOHMQuit) intervention with health system, leader (including managers and educators) and clinician components. MOHMQuit addresses a critical evidence to practice gap in the provision of smoking cessation support in antenatal care. It involves nine maternity services in New South Wales in a cluster randomised stepped-wedge controlled trial of effectiveness. This paper describes the design and rationale for the process evaluation of MOHMQuit. The process evaluation aims to assess to what extent and how MOHMQuit is being implemented (acceptability; adoption/uptake; appropriateness; feasibility; fidelity; penetration and sustainability), and the context in which it is implemented, in order to support further refinement of MOHMQuit throughout the trial, and aid understanding and interpretation of the results of the trial.

METHODS AND ANALYSIS

The process evaluation is an integral part of the stepped-wedge trial. Its design is underpinned by implementation science frameworks and adopts a mixed methods approach. Quantitative evidence from participating leaders and clinicians in our study will be used to produce individual and site-level descriptive statistics. Qualitative evidence of leaders' perceptions about the implementation will be collected using semistructured interviews and will be analysed descriptively within-site and thematically across the dataset. The process evaluation will also use publicly available data and observations from the research team implementing MOHMQuit, for example, training logs. These data will be synthesised to provide site-level as well as individual-level implementation outcomes.

ETHICS AND DISSEMINATION

The study received ethical approval from the Population Health Services Research Ethics Committee for NSW, Australia (Reference 2021/ETH00887). Results will be communicated via the study's steering committee and will also be published in peer-reviewed journals and presented at conferences.

TRIAL REGISTRATION NUMBER

Australian New Zealand Trials Registry ACTRN12622000167763. https://www.australianclinicaltrials.gov.au/anzctr/trial/ACTRN12622000167763.

Transforming nursing assessment in acute hospitals: A cluster randomised controlled trial of an evidence-based nursing core assessment (the ENCORE trial)

BACKGROUND

Patient safety is threatened when early signs of clinical deterioration are missed or not acted upon. This research began as a clinical-academic partnership established around a shared concern of nursing physical assessment practices on general wards and delayed recognition of clinical deterioration. The outcome was the development of a complex intervention facilitated at the ward level for proactive nursing surveillance.

METHODS

The evidence-based nursing core assessment (ENCORE) trial was a pragmatic cluster-randomised controlled trial. We hypothesised that ward intervention would reduce the incidence of patient rescue events (medical emergency team activations) and serious adverse events. We randomised 29 general wards in a 1:2 allocation, across 5 Australian hospitals to intervention (n = 10) and usual care wards (n = 19). Skilled facilitation over 12 months enabled practitioner-led, ward-level practice change for proactive nursing surveillance. The primary outcome was the rate of medical emergency team activations and secondary outcomes were unplanned intensive care unit admissions, on-ward resuscitations, and unexpected deaths. Outcomes were prospectively collected for 6 months following the initial 6 months of implementation. Analysis was at the patient level using generalised linear mixed models to account for clustering by ward.

RESULTS

We analysed 29,385 patient admissions to intervention (n = 11,792) and control (n = 17,593) wards. Adjusted models for overall effects suggested the intervention increased the rate of medical emergency team activations (adjusted incidence rate ratio 1.314; 95 % confidence interval 0.975, 1.773), although the confidence interval was compatible with a marginal decrease to a substantial increase in rate. Confidence intervals for secondary outcomes included a range of plausible effects from benefit to harm. However, considerable heterogeneity was observed in intervention effects by patient comorbidity. Among patients with few comorbid conditions in the intervention arm there was a lower medical emergency team activation rate and decreased odds of unexpected death. Among patients with multimorbidity in the intervention arm there were higher rates of medical emergency team activation and intensive care unit admissions.

CONCLUSION

Trial outcomes have refined our assumptions about the impact of the ENCORE intervention. The intervention appears to have protective effects for patients with low complexity where frontline teams can respond locally. It also appears to have redistributed medical emergency team activations and unplanned intensive care unit admissions, mobilising higher rates of rescue for patients with multimorbidity.

TRIAL REGISTRATION NUMBER

ACTRN12618001903279 (Date of registration: 22/11/2018; First participant recruited: 01/02/2019).

What contextual features affect the outcome and sustainability of therapeutic patient education interventions?

BACKGROUND

Therapeutic patient education interventions are influenced by contextual factors. Therefore, describing the context is crucial to understanding how it can affect therapeutic patient education interventions and contribute to outcomes. We aimed to identify the contextual features that may affect the outcome and sustainability of therapeutic patient education interventions from a healthcare professional perspective.

METHODS

Semi-structured individual interviews were conducted with healthcare professionals involved in 14 therapeutic patient education interventions covering different chronic conditions (e.g., kidney and cardiovascular diseases, chronic pain, diabetes, obesity). Interviews were recorded and fully transcribed. We followed a general inductive approach to identify themes from healthcare professionals' discourse to properly capture their perception.

RESULTS

Saturation was achieved with 28 interviews with 20 nurses, 6 dieticians, one physiotherapist and one psychologist. The average therapeutic patient education experience was 7 years. Identified contextual features clustered in 5 main themes: 1) conditions for the development of the intervention (genesis of the program: Who and what prompted it?; supports; content development; legislative framework); 2) integration of the program (in the healthcare pathway or the environment, relationship with the institution or local environment); 3) teamwork cohesion, interaction and integration with the environment (exchanges, cohesion of the team); 4) sustainability of the program; and 5) patient and healthcare professional contextual factors.

CONCLUSION

New insights into contextual features that may be involved in therapeutic patient education interventions are represented in a framework based on the Medical Research Council evaluation framework. These features need to be addressed in studies of therapeutic patient education interventions and could help healthcare professionals build more effective interventions within the context. However, describing a list of elements of the context is not enough; analyses should also focus on how the contextual elements might affect an intervention and how they interact.

Associations Between Supported Accommodation and Health and Re-offending Outcomes: a Retrospective Data Linkage Study

Following release from prison, housing and health issues form a complex and mutually reinforcing dynamic, increasing reincarceration risk. Supported accommodation aims to mitigate these post-release challenges. We describe the impact of attending Rainbow Lodge (RL), a post-release supported accommodation service for men in Sydney, Australia, on criminal justice and emergency health outcomes. Our retrospective cohort study using linked administrative data includes 415 individuals referred to RL between January 2015 and October 2020. Outcomes of interest were rates of criminal charges, emergency department (ED) presentations and ambulance attendance; and time to first reincarceration, criminal charge, ED presentation and ambulance attendance. The exposure of interest was attending RL; covariates included demographic characteristics, release year and prior criminal justice and emergency health contact. Those who attended RL (n = 170, 41%) more commonly identified as Aboriginal or Torres Strait Islander (52% vs 41%; p = 0.025). There was strong evidence that attending RL reduced the incidence criminal charges (adjusted rate ratio [ARR] = 0.56; 95% confidence interval [CI] 0.340.86; p = 0.009). Absolute rates indicate a weak protective effect of RL attendance on ED presentation and ambulance attendance; however, adjusted analyses indicated no evidence of an association between attending RL and rates of ED presentations (ARR = 0.88; 95% CI = 0.65-1.21), or ambulance attendance (ARR = 0.82; 95% CI = 0.57-1.18). There was no evidence of an association between attending RL and time to first reincarceration, charge, ED presentation or ambulance attendance. Greater detail about reasons for emergency health service contact and other self-report outcome measures may better inform how supported accommodation is meeting its intended aims.

Why and when should we cluster randomize?

A cluster randomized trial is defined as a randomized trial in which intact social units of individuals are randomized rather than individuals themselves. Outcomes are observed on individual participants within clusters (such as patients). Such a design allows assessing interventions targeting cluster-level participants (such as physicians), individual participants or both. Indeed, many interventions assessed in cluster randomized trials are actually complex ones, with distinct components targeting different levels. For a cluster-level intervention, cluster randomization is an obvious choice: the intervention is not divisible at the individual-level. For individual-level interventions, cluster randomization may nevertheless be suitable to prevent group contamination, for logistical reasons, to enhance participants' adherence, or when objectives pertain to the cluster level. An unacceptable reason for cluster randomization would be to avoid obtaining individual consent. Indeed, participants in cluster randomized trials have to be protected as in any type of trial design. Participants may be people from whom data are collected, but they may also be people who are intervened upon, and this includes both patients and physicians (for example, physicians receiving training interventions). Consent should be sought as soon as possible, although there may exist situations where participants may consent only for data collection, not for being exposed to the intervention (because, for instance, they cannot opt-out). There may even be situations where participants are not able to consent at all. In this latter situation a waiver of consent must be granted by a research ethics committee.

Exploring settings as social complex adaptive systems in setting-based health research: a scoping review

Since launching health health-promoting settings approach to health by WHO, valuable progress has happened in implementing its holistic concepts in settings such as cities, schools, workplaces, hospitals and healthcare services. However, significant knowledge-intention-success gaps still exist in creating sustainable health-promoting changes in settings. The complexity of the task of bridging this gap has contributed to the call for a complexity-informed paradigm shift to health as well as settings, followed by increasing consultation of relevant complexity theories, frameworks and tools in health research. This paper provides a critical scoping review of the application of complex adaptive system (CAS) theory in settings-based health promotion research. We included 14 papers, mostly qualitative studies, reporting on planning or implementation of change initiatives, less on its evaluation. CAS theory application was often incomplete thereby reducing the potential benefit of using this lens to understand change management. We suggest some recommendations how to comprehensively apply the CAS theory in setting-based health research and to report on all CAS characteristics to enhance the understanding of settings as adaptive health-promoting settings.

Whole-school interventions promoting student commitment to school to prevent substance use and violence, and improve educational attainment: a systematic review

Background

Whole-school interventions modify the school environment to promote health. A subset of these interventions promotes student commitment to school to prevent substance (tobacco, alcohol, other drugs) use and/or violence. A previous review identified the theory of human functioning and school organisation as a comprehensive theory of such interventions, and found evidence that these interventions reduce substance use and/or violence.

Objectives

The objectives were to search for, appraise and synthesise evidence to address the following questions: (1) What whole-school interventions promoting student commitment to school to prevent substance use and/or violence have been evaluated, what intervention subtypes are apparent and how closely do these align with the theory of human functioning and school organisation? (2) What factors relating to setting, population and intervention affect implementation? (3) What are the effects on student substance use, violence and educational attainment? (4) What is the cost-effectiveness of such interventions? (5) Are intervention effects mediated by student commitment to school or moderated by setting or population?

Data sources

A total of 56 information sources were searched (in January 2020), then an updated search of 48 of these was carried out (in May 2021). Reference lists were also searched and experts were contacted.

Review methods

Eligible studies were process/outcome evaluations of whole-school interventions to reduce student violence or substance use among students aged 5-18 years attending schools, via actions aligning with the theory of human functioning and school organisation: modifying teaching to increase engagement, enhancing student-staff relationships, revising school policies, encouraging volunteering or increasing parental involvement. Data extraction and quality assessments used existing tools. Theory and process reports were synthesised qualitatively. Outcome and economic data were synthesised narratively; outcome data were meta-analysed.

Results

Searches retrieved 63 eligible reports on 27 studies of 22 interventions. We identified four intervention subtypes focused on student participation in school-wide decisions, improving staff-student relationships, increasing engagement in learning and involving parents. The theories of change of most intervention subtypes aligned closely with the theory of human functioning and school organisation, and informed refinement of an intervention theory of change. Theories of change for interventions increasing learning engagement did not align with this theory, aiming instead to increase school commitment primarily via social skills curricula. Factors influencing the implementation included whether or not interventions were tailorable, workable and well explained. Interventions with action groups comprising staff/students, etc. and providing local data were well implemented. Implementation was also affected by whether or not schools accepted the need for change and staff had the resources for delivery. Meta-analyses suggest small, but significant, intervention effects in preventing violence victimisation and perpetration, and substance use. There was sparse and inconsistent evidence of moderation and some evidence of mediation by student commitment to school. Two economic evaluations suggested that there is the potential for the interventions to be cost-effective.

Limitations

The quality of the studies was variable and the economic synthesis was limited to two studies.

Conclusions

Whole-school interventions aiming to promote student commitment to school share similar theories of change and factors affecting implementation. They have the potential to contribute to preventing violence and substance use among young people. Future trials should aim to optimise intervention effectiveness by better theorisation, and assess implementation and effect moderators and mediators.

Study registration

This study is registered as PROSPERO CRD42019154334.

Funding

This award was funded by the National Institute for Health and Care Research (NIHR) Public Health Research programme (NIHR award ref: 17/151/05) and is published in full in Public Health Research; Vol. 12, No. 2. See the NIHR Funding and Awards website for further award information.

Promoting mental well-being in Western Australia: Act Belong Commit® mental health promotion campaign partners' perspectives

The Act Belong Commit® mental health promotion campaign aims to improve population mental well-being. Based on a social-franchising model, partnerships are sought with organizations offering 'mentally healthy activities' that foster social connectedness and mental well-being. There are four categories of partner organizations sought: Site (government agencies and health services), Associate (organizations with state-wide or national services), Community (local, grass-roots community groups) and Schools. This research explored organizational perceptions of the Act Belong Commit® Partnership Program. A purposive sample of 14 organizational representatives across all four partner categories participated in semi-structured interviews online or face-to-face between July and September 2021. Reflexive thematic analysis was used to identify three main themes: (a) A passion for promoting mental health; (b) Implementation, innovation, adaption and creation (e.g. the ability to adapt, create and deliver activities aligned with the campaign message; and (c) Future sustainability (e.g. recommendations for the sustainability of the programme). The use of reflexive thematic analysis enabled deeper insights into the complexity of the partnerships. Findings describe how the Act Belong Commit® Partnership Program supports mutually desirable objectives and extends the campaign reach. Embedded in a shared belief system that incorporates a passion for good mental health, the model supports the flexibility to adapt, create and deliver fit-for-purpose activities that promote mental well-being in the places where people live, work and play.

The core functions and forms paradigm throughout EPIS: designing and implementing an evidence-based practice with function fidelity

There are numerous frameworks for implementing evidence-based practices (EBPs) in novel settings to achieve "fidelity." However, identifying appropriate referents for fidelity poses a challenge. The Core Functions and Forms paradigm offers a model that can inform adaptation decisions throughout all phases of the Exploration, Preparation, Implementation, Sustainment (EPIS) framework. We applied the Core Functions-Forms paradigm throughout the Exploration and Preparation phases of EPIS in the design of two EBPs targeting family protective factors among Latinos in San Diego, as well as describe plans for its use in Implementation and Sustainment. We employed a distinct approach for each intervention element to contrast adaptation decisions that prioritize adherence to either form or function fidelity. We describe our application of the functions-forms paradigm within the EPIS framework, focusing on the Preparation phase. We also provide functions-forms matrices that map out the relationship between individual intervention components (forms) and the essential processes (functions) by which components are theorized to exert their impact. This case study of how the core functions-forms framework can be mapped onto EPIS can support a conceptual shift from prioritizing form fidelity to also focusing on function fidelity. This might allow interventionists to target appropriate fidelity referents when adapting an EBP, rather than defaulting to maintaining fidelity to forms as described in the protocol. We see great promise for using this framework for guiding actions throughout all EPIS phases and informing future applications of this paradigm to foster more robust fidelity to function.

Implementing a social innovation for community-based peer support for immigrant mothers in Sweden: a mixed-methods process evaluation

Introduction

A South African social innovation based on peer support for mothers was contextualized in southern Sweden. The objective of the project was to support expectant women and mothers of young children in immigrant communities to access public services that would benefit maternal and child health. This study aimed to assess how the intervention was implemented, what the contextual barriers and facilitators were, and how the implementation was perceived by those who delivered and received it.

Methods

The study used mixed methods with a convergent parallel design and followed the Medical Research Council guidance on process evaluations of complex interventions. Semi-structured interviews (n = 19) were conducted with peer supporters, client mothers, and key stakeholders involved in the intervention. The qualitative data were analyzed using content analysis. Quantitative data on peer supporters' activities were collected during contacts with client mothers and were presented descriptively.

Results

The five peer supporters had 1,294 contacts with client mothers, of which 507 were first-time contacts. The reach was perceived as wide, and the dose of the intervention was tailored to individual needs. Barriers to implementation included community mistrust of social services, norms on gender roles and parenting, and funding challenges. The implementation was facilitated by the organization's reputation, network, experience, and third-sector affiliation. Peer supporters tended to prioritize linking clients to other services over the educational components of the intervention, sometimes doing more than what was originally planned. Implementation strategies used included building trust, using multiple outreach venues, using internal support structures, and providing practical assistance as an entry point to comprehensive psychosocial support. The personal connection between peer supporters and clients was highly valued, and the building of relationships enabled them to address sensitive topics. Peer supporters sometimes experienced a blurred line between professional and personal roles.

Conclusions

Peer supporters used a variety of strategies to navigate identified barriers and facilitators. Trust was central both as a contextual factor and a strategy for implementation. It is valuable to maintain a balance between flexibility and adherence to the function of peer supporters. Further research is needed to evaluate the effects of the intervention.

Integrated Care Models for Older Adults with Depression and Physical Comorbidity: A Scoping Review

Objective

Multimorbidity is a growing challenge in the care for older people with mental illness. To address both physical and mental illnesses, integrated care management is required. The purpose of this scoping review is to identify core components of integrated care models for older adults with depression and physical comorbidity, and map reported outcomes and implementation strategies.

Methods

PubMed, EMBASE, CINAHL and Cochrane Library were searched independently by two reviewers for studies concerning integrated care interventions for older adults with depression and physical comorbidity. We used the SELFIE framework to map core components of integrated care models. Clinical and organisational outcomes were mapped.

Results

Thirty-eight studies describing thirteen care models were included. In all care models, a multidisciplinary team was involved. The following core components were mainly described: continuity, person-centredness, tailored holistic assessment, pro-activeness, treatment interaction, individualized care planning, and coordination tailored to complexity of care needs. Twenty-seven different outcomes were evaluated, with more attention given to clinical than to organisational outcomes.

Conclusion

The core components that comprise integrated care models are diverse. Future studies should focus more on implementation aspects of the intervention and describe financial parts, e.g., the cost of the intervention for the healthcare user, more transparently.

The Effects of the Jump-In Whole-School Intervention on the Weight Development of Children in Amsterdam, the Netherlands

BACKGROUND

This study assessed the effects of the "Jump-in" whole-school intervention in Amsterdam, the Netherlands, on children's weight development by comparing children exposed to the intervention and controls from 3 other large Dutch cities. Jump-in is a comprehensive intervention that aims to stimulate healthy nutrition and physical activity in children at primary schools in Amsterdam. In addition, the relationship between the intervention's implementation degree and its effectiveness was studied.

METHODS

Demographic and anthropometric data, collected by youth health care professionals via routine health checks at T0 (2014) and T1 (2019), were used to analyze possible intervention effects by comparing the weight development of children exposed to the Jump-in intervention versus unexposed controls. Implementation logs from health promotion professionals were used to determine intervention effects per implementation degree. Multilevel regression analyses were used for all analyses.

RESULTS

In total, 4299 children were included mean age ± 5.5 years (T0), 10.6 years (T1), and ≈50% boys/girls at both times. Receiving the fully implemented intervention resulted in a decline in standardized body-mass index (zBMI) compared to the controls (-0.23, confidence interval [CI] -0.33, -0.13). It also led to higher odds to move into a healthier weight category over time (odds ratio [OR] 1.36, CI 1.06, 1.74), yet no statistically significant shift towards a healthy weight was found.

CONCLUSIONS

Relative to the controls, children exposed to the intervention showed positive zBMI developments, with stronger effects when the implementation degree was higher. Despite positive results, creating more impact might require the further integration of school-based programs into whole-systems approaches that include other energy-balance behaviors.

Optimising a co-production framework for developing public health interventions: application and testing of school-based Research Action Groups

Existing frameworks for developing public health interventions have limited guidance for researchers on how to work with stakeholders to co-produce context-specific interventions. To address this, a promising framework for the co-production of public health interventions was developed through a previous review of school-based co-production. The present study aimed to optimise this co-production framework through applying and testing it in real-world secondary school contexts. Within schools, Research Action Groups (RAGs) were established with multiple school stakeholders supported by an external facilitator to use school data to produce school-specific mental health and wellbeing interventions. A mixed method process evaluation of two contextually diverse case study secondary schools in the UK was used to optimise the co-production framework. The process evaluation sought the views of those involved (students, school staff, school Senior Management Team (SMT) members, and the external co-production facilitator/researcher) on the co-production functions they were involved in so they could be modified, if needed. Data collected for the process evaluation during co-production were a researcher diary (n = 45 entries), and observations of student photography (n = 21) and RAG meetings (n = 8). Post co-production, interviews, and surveys with RAG students (n = 18) and staff (n = 8), and two school-specific SMT focus groups (n = 10) were conducted. The study identified four recommendations to optimise and integrate co-production into real world practice. They include the need to: assess schools for their readiness to undertake co-production; more effectively communicate the necessity to have stakeholders from the whole system involved; work with SMTs throughout co-production functions; involve stakeholders outside the school to support producing solutions to change school mental health and wellbeing priorities. The framework is intended to be used by researchers to integrate stakeholders into a shared decision-making process to develop interventions that meet the needs and contexts of individual schools. It could be transferred to other settings to support the development of public health interventions for other health areas, and populations. Further evaluation to test its use in other settings is needed.

Assessing the fidelity of a behavioural intervention involving academic detailing in general practice: a sub-study of the 'Implementing work-related Mental health guidelines in general PRacticE' (IMPRovE) trial

BACKGROUND

Assessing the fidelity of intervention components enables researchers to make informed judgements about the influence of those components on the observed outcome. The 'Implementing work-related Mental health guidelines in general PRacticE' (IMPRovE) trial is a hybrid III trial aiming to increase adherence to the 'Clinical Guidelines for the diagnosis and management of work-related mental health conditions in general practice'. IMPRovE is a multifaceted intervention, with one of the central components being academic detailing (AD). This study describes the fidelity to the protocol for the AD component of the IMPRovE intervention.

METHOD

All AD sessions for the trial were audio-recorded and a sample of 22% were randomly selected for fidelity assessment. Fidelity was assessed using a tailored proforma based on the Modified Conceptual Framework for fidelity assessment, measuring duration, coverage, frequency and content. A descriptive analysis was used to quantify fidelity to the protocol and a content analysis was used to elucidate qualitative aspects of fidelity.

RESULTS

A total of eight AD sessions were included in the fidelity assessment. The average fidelity score was 89.2%, ranging from 80 to 100% across the eight sessions. The sessions were on average 47 min long and addressed all of the ten chapters in the guideline. Of the guideline chapters, 9 were frequently discussed. The least frequently discussed chapter related to management of comorbid conditions. Most general practitioner (GP) participants used the AD sessions to discuss challenges with managing secondary mental conditions. In line with the protocol, opinion leaders who delivered the AD sessions largely offered evidence-based strategies aligning with the clinical guideline recommendations.

CONCLUSIONS/IMPLICATIONS

The IMPRovE AD intervention component was delivered to high fidelity. The sessions adhered to the intended duration, coverage, frequency, and content allowing participating GPs to comprehend the implementation of the guideline in their own practice. This study also demonstrates that the Modified Conceptual Fidelity Framework with a mixed methods approach can support the assessment of implementation fidelity of a behavioural intervention in general practice. The findings enhance the trustworthiness of reported outcomes from IMPRovE and show that assessing fidelity is amenable for AD and should be incorporated in other studies using AD.

TRIAL REGISTRATION

Australian New Zealand Clinical Trials Registry ACTRN 12620001163998, November 2020.

Delivery of a novel intervention to facilitate liberation from mechanical ventilation in paediatric intensive care: A process evaluation

BACKGROUND

Prolonged mechanical ventilation increases the risk of mortality and morbidity. Optimising sedation and early testing for possible liberation from invasive mechanical ventilation (IMV) has been shown to reduce time on the ventilator. Alongside a multicentre trial of sedation and ventilation weaning, we conducted a mixed method process evaluation to understand how the intervention content and delivery was linked to trial outcomes.

METHODS

10,495 children admitted to 18 paediatric intensive care units (ICUs) in the United Kingdom participated in a stepped-wedge, cluster randomised controlled trial, with 1955 clinical staff trained to deliver the intervention. The intervention comprised assessment and optimisation of sedation levels, and bedside screening of respiratory parameters to indicate readiness for a spontaneous breathing trial prior to liberation from ventilation. 193 clinical staff were interviewed towards the end of the trial. Interview data were thematically analysed, and quantitative adherence data were analysed using descriptive statistics.

RESULTS

The intervention led to a reduced duration of IMV (adjusted median difference- 7.1 hours, 95% CI -9.6 to -5.3, p = 0.01). Overall intervention adherence was 75% (range 59-85%). Ease and flexibility of the intervention promoted it use; designated responsibilities, explicit pathways of decision-making and a shared language for communication fostered proactivity and consistency towards extubation. Delivery of the intervention was hindered by established hospital and unit organisational and patient care routines, clinician preference and absence of clinical leadership.

CONCLUSIONS

The SANDWICH trial showed a significant, although small, reduction in duration of IMV. Findings suggest that greater direction in decision-making pathways, robust embedment of new practice in unit routine, and capitalising on the skills of Advanced Nurse Practitioners and physiotherapists would have contributed to greater intervention effect.

TRIAL REGISTRATION

isrctn.org Identifier: ISRCTN16998143.

Patterns in bottlenecks for implementation of health promotion interventions: a cross-sectional observational study on intervention-context interactions in the Netherlands

BACKGROUND

From a complex systems perspective, implementation should be understood as the introduction of an intervention in a context with which it needs to interact in order to achieve its function in terms of improved health. The presence of intervention-context interactions could mean that during implementation particular patterns of crucial interaction points might arise. We examined the presence of - and regularities in - such 'bottlenecks for implementation', as this could create opportunities to predict and intervene in potential implementation problems.

METHODS

We conducted a cross-sectional observational study against the background of municipal intersectoral policymaking in the Netherlands. We asked implementers of health promotion interventions to identify bottlenecks by rating the presence and importance of conditions for implementation in a range of intervention systems. We used descriptive statistics to characterize these systems (by their behaviour change method, health theme and implementation setting) and the conditions that acted as bottlenecks. After stratifying bottlenecks by intervention system and the system's characteristics, we tested our hypotheses by comparing the number and nature of the bottlenecks that emerged.

RESULTS

More than half of the possible conditions were identified as a bottleneck for implementation. Bottlenecks occurred in all categories of conditions, e.g., relating to the implementer, the intervention, and political and administrative support, and often connected with intersectoral policymaking, e.g., relating to the co-implementer and the co-implementer's organization. Both our hypotheses were supported: (1) Each intervention system came across a unique set of - a limited number of - conditions hampering implementation; (2) Most bottlenecks were associated with the characteristics of the system in which they occurred, but bottlenecks also appeared in the absence of such an association, or remained absent in the presence thereof.

CONCLUSIONS

We conclude that intervention-context interactions in integrated health policymaking may lead to both regularities and variations in bottlenecks for implementation. Regularities may partly be predicted by the function of an intervention system, and may serve as the basis for building the capacity needed for the structural changes that can bring about long-lasting health improvements. Variations may point at the need for flexibility in further tailoring the implementation approach to the - mostly unpredictable - problems at individual sites.

Online remote behavioural intervention for tics in 9- to 17-year-olds: the ORBIT RCT with embedded process and economic evaluation

Background

Behavioural therapy for tics is difficult to access, and little is known about its effectiveness when delivered online.

Objective

To investigate the clinical and cost-effectiveness of an online-delivered, therapist- and parent-supported therapy for young people with tic disorders.

Design

Single-blind, parallel-group, randomised controlled trial, with 3-month (primary end point) and 6-month post-randomisation follow-up. Participants were individually randomised (1 : 1), using on online system, with block randomisations, stratified by site. Naturalistic follow-up was conducted at 12 and 18 months post-randomisation when participants were free to access non-trial interventions. A subset of participants participated in a process evaluation.

Setting

Two hospitals (London and Nottingham) in England also accepting referrals from patient identification centres and online self-referrals.

Participants

Children aged 9-17 years (1) with Tourette syndrome or chronic tic disorder, (2) with a Yale Global Tic Severity Scale-total tic severity score of 15 or more (or > 10 with only motor or vocal tics) and (3) having not received behavioural therapy for tics in the past 12 months or started/stopped medication for tics within the past 2 months.

Interventions

Either 10 weeks of online, remotely delivered, therapist-supported exposure and response prevention therapy (intervention group) or online psychoeducation (control).

Outcome

Primary outcome: Yale Global Tic Severity Scale-total tic severity score 3 months post-randomisation; analysis done in all randomised patients for whom data were available. Secondary outcomes included low mood, anxiety, treatment satisfaction and health resource use. Quality-adjusted life-years are derived from parent-completed quality-of-life measures. All trial staff, statisticians and the chief investigator were masked to group allocation.

Results

Two hundred and twenty-four participants were randomised to the intervention (n = 112) or control (n = 112) group. Participants were mostly male (n = 177; 79%), with a mean age of 12 years. At 3 months the estimated mean difference in Yale Global Tic Severity Scale-total tic severity score between the groups adjusted for baseline and site was -2.29 points (95% confidence interval -3.86 to -0.71) in favour of therapy (effect size -0.31, 95% confidence interval -0.52 to -0.10). This effect was sustained throughout to the final follow-up at 18 months (-2.01 points, 95% confidence interval -3.86 to -0.15; effect size -0.27, 95% confidence interval -0.52 to -0.02). At 18 months the mean incremental cost per participant of the intervention compared to the control was £662 (95% confidence interval -£59 to £1384), with a mean incremental quality-adjusted life-year of 0.040 (95% confidence interval -0.004 to 0.083) per participant. The mean incremental cost per quality-adjusted life-year gained was £16,708. The intervention was acceptable and delivered with high fidelity. Parental engagement predicted child engagement and more positive clinical outcomes.

Harms

Two serious, unrelated adverse events occurred in the control group.

Limitations

We cannot separate the effects of digital online delivery and the therapy itself. The sample was predominately white and British, limiting generalisability. The design did not compare to face-to-face services.

Conclusion

Online, therapist-supported behavioural therapy for young people with tic disorders is clinically and cost-effective in reducing tics, with durable benefits extending up to 18 months.

Future work

Future work should compare online to face-to-face therapy and explore how to embed the intervention in clinical practice.

Trial registration

This trial is registered as ISRCTN70758207; ClinicalTrials.gov (NCT03483493). The trial is now complete.

Funding

This project was funded by the National Institute for Health and Care Research (NIHR) Health and Technology Assessment programme (project number 16/19/02) and will be published in full in Health and Technology Assessment; Vol. 27, No. 18. See the NIHR Journals Library website for further project information.

A qualitative evaluation of the national rollout of a diabetes prevention programme in England

BACKGROUND

The National Health Service Diabetes Prevention Programme (NHS DPP) was commissioned by NHS England in 2016 and rolled out in three 'waves' across the whole of England. It aims to help people with raised blood glucose levels reduce their risk of developing type 2 diabetes through behaviour change techniques (e.g., weight loss, dietary changes and exercise). An independent, longitudinal, mixed methods evaluation of the NHS DPP was undertaken. We report the findings from the implementation work package: a qualitative interview study with designated local leads, responsible for the local commissioning and implementation of the programme. The aim of the study was to explore how local implementation processes were enacted and adapted over time.

METHODS

We conducted a telephone interview study across two time-points. Twenty-four semi-structured interviews with local leads across 19 sampled case sites were undertaken between October 2019 and January 2020 and 13 interviews with local leads across 13 sampled case sites were conducted between July 2020 and August 2020. Interviews aimed to reflect on the experience of implementation and explore how things changed over time.

RESULTS

We identified four overarching themes to show how implementation was locally enacted and adapted across the sampled case sites: 1. Adapting to provider change; 2. Identification and referral; 3. Enhancing uptake in underserved populations; and 4. Digital and remote service options.

CONCLUSION

This paper reports how designated local leads, responsible for local implementation of the NHS DPP, adapted implementation efforts over the course of a changing national diabetes prevention programme, including how local leads adapted implementation during the COVID-19 pandemic. This paper highlights three main factors that influence implementation: the importance of facilitation, the ability (or not) to tailor interventions to local needs and the role of context in implementation.

'Health Connections': study protocol for the development of a coproduced, community-based diet, physical activity, and healthy weight intervention for UK black and Asian adults

INTRODUCTION

A limited number of diet, physical activity and weight management programmes suitable for UK black and Asian populations have been evaluated. We aim to coproduce 'Health Connections'-an ambitious new intervention to support dietary and physical activity choices, and maintaining a healthier weight, tailored to the needs of black Caribbean, black African and South Asian adults. Our existing research and public engagement work suggests that the intervention should be designed to be embedded in communities and delivered by peer educators supported by health professionals.

METHODS AND ANALYSIS

The project is underpinned by a systems perspective that posits collective efficacy within communities, behaviour change theory and coproduction. Project activities will be conducted in three stages. Stage 1: semistructured interviews will be conducted with adults from diverse South Asian ethnic groups to understand their experiences, perspectives and intervention needs, adding to our existing data from black ethnic groups. We will synthesise the data, literature, available intervention resources and local practice, and develop the theoretical framework to codevelop intervention goals, programme theory and a draft logic model of change. Stage 2: a theorised list of potential intervention components, session content and mode/s of delivery will be explored in a modified Delphi exercise and workshop to achieve consensus on the intervention format. We will also develop prototype materials and a formal implementation plan. Stage 3: a description of the intervention will be documented.

ETHICS AND DISSEMINATION

The study has received ethical approval from the School of Health Research Ethics Committee, Leeds Beckett University. Information on the project aims and voluntary participation is provided in the study participation information sheet. Consent will be certified by the completion and signing of a consent form prior to data collection. Dissemination for a range of stakeholders and audiences will include publications, presentations, short films and an infographic.

A systematic review and meta-analysis of randomised controlled trials of surgical treatments for ingrown toenails part II: healing time, post-operative complications, pain, and participant satisfaction

BACKGROUND

When performing nail surgery, clinicians must choose from a multitude of procedures and variations within each procedure. Much has been published to guide this decision making, but there are a lack of up to date robust systematic reviews to assess the totality of this evidence.

METHODS

Five databases (MEDLINE, Embase, CINAHL, Web of Science and CENTRAL) and two registers (Clinicaltrials.gov and ISRCTN) were searched to January 2022 for randomised trials evaluating the effects of a surgical intervention(s) for ingrown toenails. Two independent reviewers screened records, extracted data, assessed risk of bias and certainty of evidence. Data on co-primary outcomes of symptom relief and symptomatic regrowth were presented in our first paper. This paper presents data for the secondary outcomes and further discussion.

RESULTS

Of 3,928 records identified, 36 randomised trials were included in the systematic review. Healing time appears to be reduced with shorter application of phenol. A reduced healing time was also apparent was with the addition of curettage, although this may also increase the risk of post-operative bleeding and pain. Post operative bleeding was also reportedly lower in people who received local anaesthetic with epinephrine but no tourniquet. Use of phenol with nail bed excision may decrease the risk of infection. Lower pain scores were reported when using partial matrixectomy and surgical interventions with phenol. Shorter duration of pain was reported with phenolisation and wedge resection. Participant satisfaction was high overall.

CONCLUSION

This second paper reports secondary outcomes from a robust systematic review of randomised trials on surgical treatment of ingrown toenails. Despite the large volume of clinical trials conducted on the topic, few clinical conclusions can be drawn due to the poor quality of these studies. Further high-quality clinical trials are needed to answer fundamental questions in the surgical treatment of ingrown toenails.

Context matters-the phased development of an adaptable food literacy intervention: Up for Cooking

Adapting interventions to the context increases the impact and sustainability of interventions. Literature acknowledges the need to adapt existing interventions and that these adaptations should be clearly reported. However, little is known about how to incorporate adaptation from the beginning. This paper argues that interventions should be developed and adaptations should be made using an ongoing non-linear approach. An action-oriented research approach with feedback loops is proposed. We illustrate this with the development of a food literacy intervention 'Up for Cooking' (Dutch: Zin in Koken) and present lessons learned in developing, implementing and studying such adaptable interventions. Interventions should clearly define and differentiate the intervention function and form. Implementers, in turn, should be encouraged to tailor interventions within a form that fits with a specific context. Sufficient time, continuous adaptation based on co-creation, feedback loops and interdisciplinary collaboration are important prerequisites for the development of adaptable interventions.

Scaling-up and future sustainability of a national reproductive genetic carrier screening program

An understanding of factors influencing implementation is essential to realise the benefits of population-based reproductive genetic carrier screening programs. The aim of this study was to synthesise data collected during the Australian Reproductive Genetic Carrier Screening Project (Mackenzie's Mission) to track how priorities shifted over time and identify important factors during scaling-up and for sustainment. We used a multi-method qualitative approach to integrate longitudinal project data collected from 10 project committees with 16 semi-structured interviews conducted with study team members. Both datasets were analysed using the Consolidated Framework for Implementation Research (CFIR) to identify constructs of interest within early, mid-point, and future implementation phases. Several CFIR constructs were present across implementation. The complexity of implementation presented challenges that were overcome through a quality-designed and packaged product, formal and informal networks and communication, and access to knowledge and information. Addressing the diverse consumer needs through resources and increasing community and non-genetic speciality engagement remained a priority throughout and for future sustainment. Going forward, further addressing program complexities and securing funding were emphasised. By applying an implementation framework, findings from this study may be useful for future effort towards building and/or sustaining reproductive genetic carrier screening programs.

Aligning intuition and theory: a novel approach to identifying the determinants of behaviours necessary to support implementation of evidence into practice

BACKGROUND

Disentangling the interplay between experience-based intuition and theory-informed implementation is crucial for identifying the direct contribution theory can make for generating behaviour changes needed for successful evidence translation. In the context of 'clinicogenomics', a complex and rapidly evolving field demanding swift practice change, we aimed to (a) describe a combined clinician intuition- and theory-driven method for identifying determinants of and strategies for implementing clinicogenomics, and (b) articulate a structured approach to standardise hypothesised behavioural pathways and make potential underlying theory explicit.

METHODS

Interview data from 16 non-genetic medical specialists using genomics in practice identified three target behaviour areas across the testing process: (1) identifying patients, (2) test ordering and reporting, (3) communicating results. The Theoretical Domains Framework (TDF) was used to group barriers and facilitators to performing these actions. Barriers were grouped by distinct TDF domains, with 'overarching' TDF themes identified for overlapping barriers. Clinician intuitively-derived implementation strategies were matched with corresponding barriers, and retrospectively coded against behaviour change techniques (BCTs). Where no intuitive strategies were provided, theory-driven strategies were generated. An algorithm was developed and applied to articulate how implementation strategies address barriers to influence behaviour change.

RESULTS

Across all target behaviour areas, 32 identified barriers were coded across seven distinct TDF domains and eight overarching TDF themes. Within the 29 intuitive strategies, 21 BCTs were represented and used on 49 occasions to address 23 barriers. On 10 (20%) of these occasions, existing empirical links were found between BCTs and corresponding distinct TDF-coded barriers. Twenty additional theory-driven implementation strategies (using 19 BCTs on 31 occasions) were developed to address nine remaining barriers.

CONCLUSION

Clinicians naturally generate their own solutions when implementing clinical interventions, and in this clinicogenomics example these intuitive strategies aligned with theoretical recommendations 20% of the time. We have matched intuitive strategies with theory-driven BCTs to make potential underlying theory explicit through proposed structured hypothesised causal pathways. Transparency and efficiency are enhanced, providing a novel method to identify determinants of implementation. Operationalising this approach to support the design of implementation strategies may optimise practice change in response to rapidly evolving scientific advances requiring swift translation into healthcare.

The exploration of remote simulation strategies for the acquisition of psychomotor skills in medicine: a pilot randomized controlled trial

Background

Progress in remote educational strategies was fueled by the advent of the COVID-19 pandemic. This pilot RCT explored the efficacy of a decentralized model of simulation based on principles of observational and peer-to-peer learning for the acquisition of surgical skills.

Methods

Sixty medical students from the University of Montreal learned the running subcuticular suture in four different conditions: (1) Control group (2) Self-learning (3) Peer-learning (4) Peer-learning with expert feedback. The control group learned with error-free videos, while the others, through videos illustrating strategic sub-optimal performances to be identified and discussed by students. Performance on a simulator at the end of the learning period, was assessed by an expert using a global rating scale (GRS) and checklist (CL).

Results

Students engaging in peer-to-peer learning strategies outperformed students who learned alone. The presence of an expert, and passive vs active observational learning strategies did not impact performance.

Conclusion

This study supports the efficacy of a remote learning strategy and demonstrates how collaborative discourse optimizes the students' acquisition of surgical skills. These remote simulation strategies create the potential for implantation in future medical curriculum design.Trial Registration: NCT04425499 2020-05-06.

Connecting the science and practice of implementation - applying the lens of context to inform study design in implementation research

The saying "horses for courses" refers to the idea that different people and things possess different skills or qualities that are appropriate in different situations. In this paper, we apply the analogy of "horses for courses" to stimulate a debate about how and why we need to get better at selecting appropriate implementation research methods that take account of the context in which implementation occurs. To ensure that implementation research achieves its intended purpose of enhancing the uptake of research-informed evidence in policy and practice, we start from a position that implementation research should be explicitly connected to implementation practice. Building on our collective experience as implementation researchers, implementation practitioners (users of implementation research), implementation facilitators and implementation educators and subsequent deliberations with an international, inter-disciplinary group involved in practising and studying implementation, we present a discussion paper with practical suggestions that aim to inform more practice-relevant implementation research.

Non-Positive Experiences Encountered by Pupils During Participation in a Mindfulness-Informed School-Based Intervention

Mindfulness-informed school-based mental health curricula show much promise in cultivating a positive school climate which supports the well-being and mental health of pupils and staff. However, non-positive pupil outcomes and experiences of school-based mental health interventions are often under-recognised and under-reported. This study sought to capture non-positive pupil experiences of a popular mindfulness-informed curriculum. Some pupils across all schools in the study described non-positive experiences, including having troubling thoughts and emotions, and not finding the programme effective. Contexts surrounding these experiences are explored and linked to existing literature, and subsequent recommendations for improvements are made, including the importance of having clear programme structure, definitions and aims, acknowledging and accommodating fidelity issues as best as possible, and better highlighting the potential for non-positive experiences and how they may be reduced.

Rapid-cycle designs to adapt interventions for COVID-19 in safety-net healthcare systems

Racial/ethnic minority, low socioeconomic status, and rural populations are disproportionately affected by COVID-19. Developing and evaluating interventions to address COVID-19 testing and vaccination among these populations are crucial to improving health inequities. The purpose of this paper is to describe the application of a rapid-cycle design and adaptation process from an ongoing trial to address COVID-19 among safety-net healthcare system patients. The rapid-cycle design and adaptation process included: (a) assessing context and determining relevant models/frameworks; (b) determining core and modifiable components of interventions; and (c) conducting iterative adaptations using Plan-Do-Study-Act (PDSA) cycles. PDSA cycles included: Plan. Gather information from potential adopters/implementers (e.g., Community Health Center [CHC] staff/patients) and design initial interventions; Do. Implement interventions in single CHC or patient cohort; Study. Examine process, outcome, and context data (e.g., infection rates); and, Act. If necessary, refine interventions based on process and outcome data, then disseminate interventions to other CHCs and patient cohorts. Seven CHC systems with 26 clinics participated in the trial. Rapid-cycle, PDSA-based adaptations were made to adapt to evolving COVID-19-related needs. Near real-time data used for adaptation included data on infection hot spots, CHC capacity, stakeholder priorities, local/national policies, and testing/vaccine availability. Adaptations included those to study design, intervention content, and intervention cohorts. Decision-making included multiple stakeholders (e.g., State Department of Health, Primary Care Association, CHCs, patients, researchers). Rapid-cycle designs may improve the relevance and timeliness of interventions for CHCs and other settings that provide care to populations experiencing health inequities, and for rapidly evolving healthcare challenges such as COVID-19.

Comparative Assessment of Diet Quality and Adherence to a Structured Nutrition and Exercise Intervention Compared with Usual Care in Pregnancy in a Randomized Trial

Background

In trials testing the efficacy of diet and exercise modifications during pregnancy on health outcomes, assessment of participant adherence to interventions of diet and exercise is rarely reported, with few standard methods existing to measure adherence.

Objective

We aimed to assess the maternal diet quality and create an algorithm to evaluate adherence to an intervention of high protein/dairy nutrition and walking exercise from early pregnancy to birth.

Methods

In Be Healthy in Pregnancy randomized trial (NCT01693510), diet quality was measured using scores from an adapted PrimeScreen food frequency questionnaire, nutrient intake assessed by 3-day diet records, and physical activity using accelerometry at 14-17 (early), 26-28 (middle), and 36-38 (late) weeks' gestation. A novel adherence score was derived by combining data for compliance with prescribed protein and energy intakes and daily step counts in the intervention group. Between-group diet quality scores and changes in adherence scores in the intervention group across pregnancy were analyzed using generalized estimating equations adjusted for prepregnancy body mass index and study site.

Results

Diet scores were similar for intervention (n = 55) and control (n = 56) groups at baseline but only the intervention group significantly improved and maintained their scores from early to middle (18.7 ± 7.6 vs. 22.9 ± 6.1; P < 0.001) and late (22.5 ± 6.9; P < 0.008) pregnancy. Protein intake was significantly (P < 0.001) higher but energy intakes were similar in the intervention group compared with those in the control group. Adherence scores for the intervention increased significantly (P < 0.01) from early (1.52 ± 0.70) to midpregnancy (1.89 ± 0.82) but declined from midpregnancy to late (1.55 ± 0.78; P < 0.0005) pregnancy primarily owing to lower step counts.

Conclusions

Adherence to an intervention may decline toward the end of pregnancy, particularly in maintaining physical activity. Creation of adherence scores is a feasible approach to measure combined intervention compliance for diet and physical activity and may increase transparency in interpreting results of randomized trials in pregnancy.This trial was registered at clinicaltrials.gov as NCT01689961 (https://clinicaltrials.gov/ct2/show/NCT01689961?cond=NCT01689961&rank=1; registered on 21 September 2012).

The Implementation Experience of COVID-19 Rapid Antigen Testing in a Large-Scale Construction Project in Victoria, Australia

The coronavirus (COVID-19) pandemic has caused major disruptions to industries and workplaces. Rapid Antigen Tests (RATs) for COVID-19, which allow individuals to self-administer tests and receive timely results without laboratory testing, provide the opportunity for surveillance testing of asymptomatic individuals in non-medical settings. However, the literature offers few lessons regarding how to create enabling conditions for effective and sustainable implementation in a workplace setting. Guided by the RE-AIM framework, we assessed factors associated with the adoption, implementation, and maintenance of mandatory RAT in a large-scale construction project in Victoria, Australia. We used a mixed methods approach involving site observation, worker surveys (n = 30), and interviews with 51 site workers and managers to understand the implementation experience. Factors which facilitated adoption included easy, non-invasive testing procedure; sense of workplace safety; and strong backing by management and acceptance by workers that RATs helped limit COVID-19-related lost days of work. Gaps in knowledge and adherence to testing protocols, logistical challenges (test kit supply, observation of test results), and low appetite for long-term, mandatory testing emerged as challenges for effective implementation and sustainability. As RAT becomes normalized in a range of workplace settings, strategies will be required to support the sustainability of implementation, including longer-term acceptability of surveillance testing and adherence to testing protocols.

Supplementary Information

The online version contains supplementary material available at 10.1007/s43477-023-00085-4.

Effects of interdisciplinary pain rehabilitation programs on neuropathic and non-neuropathic chronic pain conditions - a registry-based cohort study from Swedish Quality Registry for Pain Rehabilitation (SQRP)

BACKGROUND AND AIM

Neuropathic pain arises as a direct consequence of a lesion or disease affecting the somatosensory system. Pharmacological treatments for neuropathic pain often fail despite following guidelines. Interdisciplinary Pain Rehabilitation Programs (IPRP) are an effective intervention for chronic pain conditions. Little research has investigated whether IPRP can benefit patients with chronic neuropathic pain compared to other chronic pain conditions. This study assesses the real-world effects of IPRP on patients with chronic neuropathic pain compared to non-neuropathic patients using Patient-Reported Outcome Measures (PROMs) available in the Swedish Quality Registry for Pain Rehabilitation (SQRP).

METHODS

A neuropathic group of patients (n = 1,654) were identified in two steps. This group was compared to a non-neuropathic group (n = 14,355) composed of common diagnoses (low back pain, fibromyalgia, whiplash associated disorders, and Ehlers-Danlos Syndrome) in relation to background variables, three overall outcome variables, and mandatory outcome variables (pain intensity, psychological distress symptoms, activity/participation aspects and health-related quality of life variables). Of these patients 43-44% participated in IPRP.

RESULTS

At assessment, the neuropathic group reported significantly (with small effect sizes (ES)) more physician visits the previous year, older age, shorter pain durations, and less spatial extent of the pain (moderate ES). Moreover, for the 22 mandatory outcome variables, we found only clinically insignificant differences according to ESs between the groups. For patients participating in IPRP, the neuropathic group displayed equal or in some cases slightly superior results compared to the non-neuropathic group.

DISCUSSION AND CONCLUSION

After assessing the real-world effects of IPRP, this large study found that neuropathic pain patients can benefit from the IPRP intervention. Both registry studies and RCTs are needed to better understand which patients with neuropathic pain are most suitable for IPRP and to what extent special considerations need to be made for these patients within the framework of IPRP.

[Development and evaluation of complex interventions in nursing : Application of the medical research council's framework using the example of interventions to prevent physical restraints]

Many nursing interventions are complex. They comprise different intervention elements (components) and aim to change processes or behaviours of individuals or groups. A framework of the British Medical Research Council comprises methodological recommendations for the development and evaluation of complex interventions. This narrative review describes the framework's methodological recommendations using an example of interventions to reduce physical restraints in hospital and long-term care settings, such as bedrails or belts in chairs and beds. In addition to the characteristics of the complex interventions, the development and theoretical foundation of the interventions as well as the feasibility test and evaluation is described.

System Action Learning: Reorientating Practice for System Change in Preventive Health

It is now widely accepted that many of the problems we face in public health are complex, from chronic disease to COVID-19. To grapple with such complexity, researchers have turned to both complexity science and systems thinking to better understand the problems and their context. Less work, however, has focused on the nature of complex solutions, or intervention design, when tackling complex problems. This paper explores the nature of system intervention design through case illustrations of system action learning from a large systems level chronic disease prevention study in Australia. The research team worked with community partners in the design and implementation of a process of system action learning designed to reflect on existing initiatives and to reorient practice towards responses informed by system level insights and action. We were able to observe and document changes in the mental models and actions of practitioners and in doing so shine a light on what may be possible once we turn our attention to the nature and practice of system interventions.

Common Elements Approaches to Implementation Research and Practice: Methods and Integration with Intervention Science

We propose that common elements approaches can advance implementation research and practice and facilitate pragmatic use of intervention and implementation evidence. Common elements are practices or processes frequently shared by interventions or implementations. Traditional common elements methodologies use synthesis, distillation, and statistics to describe and evaluate the merit of common ingredients in effective interventions. Recent developments include identifying and testing common configurations of elements, processes, and context variables across the literature of effective interventions and implementations. While common elements thinking has grown popular in intervention science, it has rarely been utilized in implementation science, and specifically, combined with the intervention literature. The goals of this conceptual methodology paper are to (1) provide an overview of the common elements concept and how it may advance implementation research and usability for practice, (2) give a step-by-step guide to systematic common elements reviews that synthesizes and distills the intervention and implementation literature together, and (3) offer recommendations for advancing element-level evidence in implementation science. A narrative review of the common elements literature was conducted with attention to applications to implementation research. A six-step guide to using an advanced common elements methodology was provided. Examples of potential results are presented, along with a review of the implications for implementation research and practice. Finally, we reviewed methodological limitations in current common elements approaches, and identified steps towards realizing their potential. Common elements methodologies can (a) synthesize and distill the implementation science literature into practical applications, (b) generate evidence-informed hypotheses about key elements and determinants in implementation and intervention processes and mechanisms, and (c) promote evidence-informed precision tailoring of intervention and implementation to context. To realize this potential, common elements approaches need improved reporting of details from both successful and unsuccessful intervention and implementation research, more data availability, and more testing and investigation of causal processes and mechanisms of change from diverse theories.

Supplementary Information

The online version contains supplementary material available at 10.1007/s43477-023-00077-4.

Treatment fidelity in a pragmatic clinical trial of music therapy for premature infants and their parents: the LongSTEP study

BACKGROUND

Treatment fidelity (TF) refers to methodological strategies used to monitor and enhance the reliability and validity of interventions. We evaluated TF in a pragmatic RCT of music therapy (MT) for premature infants and their parents.

METHODS

Two hundred thirteen families from seven neonatal intensive care units (NICUs) were randomized to receive standard care, or standard care plus MT during hospitalization, and/or during a 6-month period post-discharge. Eleven music therapists delivered the intervention. Audio and video recordings from sessions representing approximately 10% of each therapists' participants were evaluated by two external raters and the corresponding therapist using TF questionnaires designed for the study (treatment delivery (TD)). Parents evaluated their experience with MT at the 6-month assessment with a corresponding questionnaire (treatment receipt (TR)). All items as well as composite scores (mean scores across items) were Likert scales from 0 (completely disagree) to 6 (completely agree). A threshold for satisfactory TF scores (≥4) was used in the additional analysis of dichotomized items.

RESULTS

Internal consistency evaluated with Cronbach's alpha was good for all TF questionnaires (α ≥ 0.70), except the external rater NICU questionnaire where it was slightly lower (α 0.66). Interrater reliability measured by intraclass correlation coefficient (ICC) was moderate (NICU 0.43 (CI 0.27, 0.58), post-discharge 0.57 (CI 0.39, 0.73)). Gwet's AC for the dichotomized items varied between 0.32 (CI 0.10, 0.54) and 0.72 (CI 0.55, 0.89). Seventy-two NICU and 40 follow-up sessions with 39 participants were evaluated. Therapists' mean (SD) TD composite score was 4.88 (0.92) in the NICU phase and 4.95 (1.05) in the post-discharge phase. TR was evaluated by 138 parents. The mean (SD) score across intervention conditions was 5.66 (0.50).

CONCLUSIONS

TF questionnaires developed to assess MT in neonatal care showed good internal consistency and moderate interrater reliability. TF scores indicated that therapists across countries successfully implemented MT in accordance with the protocol. The high treatment receipt scores indicate that parents received the intervention as intended. Future research in this area should aim to improve the interrater reliability of TF measures by additional training of raters and improved operational definitions of items.

TRIAL REGISTRATION

Longitudinal Study of music Therapy's Effectiveness for Premature infants and their caregivers - "LongSTEP".

CLINICALTRIALS

gov Identifier: NCT03564184. Registered on June 20, 2018.

Building the foundation for a community-generated national research blueprint for inherited bleeding disorders: research priorities in health services; diversity, equity, and inclusion; and implementation science

BACKGROUND

The National Hemophilia Foundation (NHF) conducted extensive all-stakeholder inherited bleeding disorder (BD) community consultations to inform a blueprint for future research. Sustaining and expanding the specialized and comprehensive Hemophilia Treatment Center care model, to better serve all people with inherited BDs (PWIBD), and increasing equitable access to optimal health emerged as top priorities.

RESEARCH DESIGN AND METHODS

NHF, with the American Thrombosis and Hemostasis Network (ATHN), convened multidisciplinary expert working groups (WG) to distill priority research initiatives from consultation findings. WG5 was charged with prioritizing health services research (HSR); diversity, equity, and inclusion (DEI); and implementation science (IS) research initiatives to advance community-identified priorities.

RESULTS

WG5 identified multiple priority research themes and initiatives essential to capitalizing on this potential. Formative studies using qualitative and mixed methods approaches should be conducted to characterize issues and meaningfully investigate interventions. Investment in HSR, DEI and IS education, training, and workforce development are vital.

CONCLUSIONS

An enormous amount of work is required in the areas of HSR, DEI, and IS, which have received inadequate attention in inherited BDs. This research has great potential to evolve the experiences of PWIBD, deliver transformational community-based care, and advance health equity.

Generalizability of randomized controlled trials in primary health care: Applying the PRECIS-2 tool on published protocols

RATIONALE

Pragmatic design may facilitate the generalizability of effectiveness of randomized controlled trials (RCTs) in primary health care (PHC).

AIMS AND OBJECTIVES

The aim of this study was to investigate whether published protocols in PHC were designed pragmatically and to explore whether specific trial characteristics may be related to a pragmatic design.

METHODS

Using the Pragmatic Explanatory Continuum Indicator Summary-2 (PRECIS-2), we assessed pragmatism for 123 published RCT protocols. For each domain, we calculated the mean score with the 95% confidence interval (95% CI). Interrater reliability was assessed by weighted κ-coefficient with 95% CI. We examined potential associations of published protocol characteristics with overall pragmatism by performing univariate and multivariate analyses.

RESULTS

We observed the highest score for primary analysis (4.66, 95% CI: 4.51, 4.82). The eligibility score was intermediate (3.16, 95% CI: 3.01, 3.32). Domains with scores towards the explanatory side included organization (2.50, 95% CI: 2.36, 2.63), flexibility of delivery (2.74, 95% CI: 2.60, 2.88) and flexibility of adherence (3.00, 95% CI: 2.83, 3.17). Interrater agreement was good (κ = 0.61; 95% CI: 0.34, 0.80; p < 0.001). Higher sample sizes were correlated to a pragmatic design (odds ratio: 6.86, 95% CI: 1.64, 28.75; p = 0.04).

CONCLUSION

Most RCT protocols were rated as intermediate in the pragmatic-explanatory continuum. Future research may guide all stakeholders on how best to incorporate the level of pragmatism in the interpretation of the results so that the trials are more likely to be applicable in real-world settings.

Characterizing modifications to a comparative effectiveness research study: the OPTIMIZE trial-using the Framework for Reporting Adaptations and Modifications to Evidence-based Interventions (FRAME)

BACKGROUND

The OPTIMIZE trial is a multi-site, comparative effectiveness research (CER) study that uses a Sequential Multiple Assessment Randomized Trial (SMART) designed to examine the effectiveness of complex health interventions (cognitive behavioral therapy, physical therapy, and mindfulness) for adults with chronic low back pain. Modifications are anticipated when implementing complex interventions in CER. Disruptions due to COVID have created unanticipated challenges also requiring modifications. Recent methodologic standards for CER studies emphasize that fully characterizing modifications made is necessary to interpret and implement trial results. The purpose of this paper is to outline the modifications made to the OPTIMIZE trial using the Framework for Reporting Adaptations and Modifications to Evidence-Based Interventions (FRAME) to characterize modifications to the OPTIMIZE trial in response to the COVID pandemic and other challenges encountered.

METHODS

The FRAME outlines a strategy to identify and report modifications to evidence-based interventions or implementation strategies, whether planned or unplanned. We use the FRAME to characterize the process used to modify the aspects of the OPTIMIZE trial. Modifications were made to improve lower-than-anticipated rates of treatment initiation and COVID-related restrictions. Contextual modifications were made to permit telehealth delivery of treatments originally designed for in-person delivery. Training modifications were made with study personnel to provide more detailed information to potential participants, use motivational interviewing communication techniques to clarify potential participants' motivation and possible barriers to initiating treatment, and provide greater assistance with scheduling of assigned treatments.

RESULTS

Modifications were developed with input from the trial's patient and stakeholder advisory panels. The goals of the modifications were to improve trial feasibility without compromising the interventions' core functions. Modifications were approved by the study funder and the trial steering committee.

CONCLUSIONS

Full and transparent reporting of modifications to clinical trials, whether planned or unplanned, is critical for interpreting the trial's eventual results and considering future implementation efforts.

TRIAL REGISTRATION

ClinicalTrials.gov NCT03859713. Registered on March 1, 2019.