How pragmatic is it? Lessons learned using PRECIS and RE-AIM for determining pragmatic characteristics of research

Programme science methodologies and practices that address "FURRIE" challenges: examples from the field

INTRODUCTION

"Programme science" deploys scientific methods to address questions that are a priority to support the impact of public health programmes. As such, programme science responds to the challenges of making such studies: (1) feasible to undertake, (2) useful, (3) rigorous, (4) real-world-relevant, (5) informative, and undertaken by (6) equitable partnerships. The acronym "FURRIE" is proposed to describe this set of six challenges. This paper discusses selected HIV/STI (sexually transmitted infection) programme science case studies to illustrate how programme science rises to the FURRIE challenges.

DISCUSSION

One way in which programme science is made more feasible is through the analysis and interpretation of data collected through service delivery. For some questions, these data can be augmented through methods that reach potential clients of services who have not accessed services or been lost to follow-up. Process evaluation can enhance the usefulness of programme science by studying implementation processes, programme-client interactions and contextual factors. Ensuring rigour by limiting bias and confounding in the real-world context of programme science studies requires methodological innovation. Striving for scientific rigour can also have the unintended consequence of creating a gap between what happens in a study, and what happens in the "real-world." Community-led monitoring is one approach to grounding data collection in the real-world experience of clients. Evaluating complex, context-specific strategies to strengthen health outcomes in a way that is informative for other settings requires clear specification of the intervention packages that are planned and delivered in practice. Programme science provides a model for equitable partnership through co-leadership between programmes, researchers and the communities they serve.

CONCLUSIONS

Programme science addresses the FURRIE challenges, thereby improving programme impact and ultimately health outcomes and health equity. The adoption and adaptation of the types of novel programme science approaches showcased here should be promoted within and beyond the HIV/STI field.

Identification of weight loss interventions for translation among endometrial cancer survivors: A RE-AIM analysis

Interventions for obesity-related cancers that combine nutrition and physical activity for weight loss exist; however, their application to survivors of endometrial cancer is unknown. Furthermore, little is known about pre-implementation perceptions of existing programs from a variety of interested persons (physicians, researchers) who may be part of the implementation team. Adapting an existing intervention rather than developing a new intervention may speed the translational lag time as long as intervention characteristics and fit within the delivery system are considered during the planning phase. To describe the process of determining the core elements of obesity-related interventions for cancer survivors and determine which one might be best delivered by an urban healthcare system that predominantly serves individuals who live in rural areas of Virginia and West Virginia. A pragmatic review of the literature was conducted via PubMed and Google Scholar with broad search terms of cancer survivor AND weight loss AND health intervention. Identified interventions were scored related to the Practical, Robust Implementation and Sustainability Model-which is an extension of RE-AIM framework to guide the understanding of who, what, where, when, and how the intervention was conducted. Intervention characteristics are reported. In addition, ratings from three independent reviewers on the validated 5-point Likert scale of an intervention's acceptability, appropriateness, and feasibility in the intended delivery system were collected and summarized. Twelve interventions were identified with an average sample size of 241(±195) and a range of 48-683 participants. Target populations included survivors of colorectal, breast, and endometrial cancers as well as general cancer survivors and included both men and women or only women. Most participants (74%) identified as white/Caucasian and average age ranged from 47.1 to 65.9 years. Program duration ranged from 4 weeks to 18 months, with an average duration of 32 weeks. Intervention dosage ranged from three times a week to once a month. Intervention acceptability, appropriateness, and feasibility had average and standard deviation ratings of 3.52(±0.46), 3.41(±0.45), and 3.21(±0.46), respectively, out of 5. The four interventions with the highest combined acceptable, appropriate, and feasible scores are being considered for potential use as an obesity-related intervention for survivors of endometrial cancer. Future work is needed to determine relevant adaptations and efficacy among survivors of endometrial cancer with obesity. Our approach may be beneficial for other interventionists aiming to speed intervention development and implementation.

Fidelity, pragmatism and the "grey line" in between-exploring the delivery of a pragmatic physical activity randomised controlled trial-a secondary analysis

BACKGROUND

Intervention fidelity in health services research has been poor with a reported lack of understanding about what constitutes pragmatic adaptation of interventions and what constitutes failure to maintain intervention fidelity. However, the challenges facing those delivering such interventions have not been thoroughly explored. The aims of this study were to critically explore the challenges in maintaining fidelity experienced by physiotherapy staff and support workers when delivering a complex intervention for older people living with frailty.

METHODS

This study is a secondary analysis of data from a process evaluation of a large randomised controlled trial (RCT). The process evaluation employed qualitative methodologies with mixed methods including a variety of data collection methods, including participant observation, semi-structured interviews and documentary analysis. Thematic analysis was used to make sense of the data.

RESULTS

Many therapy staff felt ongoing confusion about what was acceptable to adapt and what needed to follow the protocol exactly. We found that some therapy staff were able to embrace the challenges of pragmatically adapting interventions while maintaining intervention fidelity, others stuck rigidly to the protocol and failed to adapt interventions where it was necessary.

CONCLUSION

It was clear that the understanding of fidelity and pragmatism was poor. While pragmatic trials are vital to replicate real world clinical practice, further guidance may need to be developed in order to guide the level of adaptation that is acceptable before fidelity is undermined.

Chiropractic attitude and utilisation of evidence-based practice in South Africa: a secondary analysis

BACKGROUND

Evidence-Based Practice (EBP) is a model utilised by the majority of healthcare professionals and is a clinical framework that assists with decision-making related to patient care, to improve outcomes and patient satisfaction. The study aimed to analyse the attitudes, skills, and utilization of evidence-based practice (EBP) among South African chiropractors, focusing on perceived skill levels, training, use and identifying facilitators and barriers to EBP application.

METHODS

A descriptive cross-sectional quantitative secondary analysis was conducted by inviting registered chiropractors in South Africa (n = 920) to participate in an anonymous online questionnaire using the Evidence-Based Practice Attitude and Utilisation Survey (EBASE).

RESULTS

A total of 132 chiropractors completed the survey, yielding a response rate of 14.4%. Of the respondents, 59.9% were female, 52.3% were between 26 and 35 years old, and 63.3% had graduated from the University of Johannesburg. A third of respondents stated they have poor clinical research skills. Over half of the respondents (56.8%) indicated that EBP constituted a significant part of their education except for completing systematic reviews or meta-analyses. Published clinical evidence was ranked 6th as a source of information for clinical decisions. The obstacles indicated were time constraints and a lack of clinical research in complementary and alternative medicine. Access to the internet, databases and research tools were facilitators that were deemed to be "very useful" in promoting EBP.

CONCLUSION

The majority of South African chiropractors are generally favourable towards EBP, and this practice therefore appears to be utilised and embraced, with the requisite skills.

Secondary prevention by structured semi-interactive stroke prevention package in INDIA (SPRINT INDIA): Findings from the process evaluation of a randomized controlled trial

INTRODUCTION

Secondary Prevention by Structured Semi-Interactive Stroke Prevention Package in INDIA Trial delivered secondary stroke awareness intervention to sub-acute stroke patients in form of workbook, videos and SMS across 31 centres in 12 languages. Trial was stopped for futility due to fewer vascular outcomes than anticipated. Trial results indicated that trial intervention, did not lead to reduction in vascular events. We carried out process evaluation, to evaluate trial implementation and participant's perspectives, to comprehend the trial's futile outcomes.

MATERIALS AND METHODS

Using mixed methods approach, qualitative interviews and quantitative data from case report forms, workbooks and questionnaires were analysed to measure intervention fidelity and contamination. Using purposive sampling, 115 interviews of patient-caregiver dyads and health professionals at 11 centres and 2 focus group discussions were held.

RESULTS AND DISCUSSION

Iterative thematic analysis of qualitative data was done with RE-AIM and realist models. There was good fidelity to intervention and adherence to protocol; however, there was dilution of inclusion criteria by randomly enrolling uneducated and caregiver-dependent patients. Centre coordinators provided counselling to both arms, not specified by protocol, causing bias. Coordinators found it difficult to keep patients motivated to view intervention which was corroborated by fidelity questionnaire showing decreased viewing of intervention for a year. Cardiovascular protection improved in routine care by virtue of participating in trial. No contamination of intervention was reported.

CONCLUSION

The intervention was acceptable by patients and caregivers, which could be made a community-based programme. Reasons identified for decreased viewing were repetitive content and non-availability of personal cellular device.

The potential impact of clinical decision support on nonwaivered primary care clinicians' prescribing of buprenorphine

Elimination of the X-waiver increased potential buprenorphine prescribers 13-fold, but growth in prescribing will likely be much lower. We explored self-assessments of nonwaivered primary care clinicians (PCCs) for factors affecting their likelihood to prescribe buprenorphine were the X-waiver eliminated (since realized January 2023) and the potential impacts of a clinical decision-support (CDS) tool for opioid use disorder (OUD). Cross-sectional survey data were obtained between January 2021 and March 2022 from 305 nonwaivered PCCs at 3 health systems. Factors explored were patient requests for buprenorphine, PCC access to an OUD-CDS, and PCC confidence and abilities for 5 OUD-care activities. Relationships were described using descriptive statistics and odds ratios. Only 26% of PCCs were more likely to prescribe buprenorphine upon patient request, whereas 63% were more likely to prescribe with the OUD-CDS. PCC confidence and abilities for some OUD-care activities were associated with increased prescribing likelihood from patient requests, but none were associated with the OUD-CDS. The OUD-CDS may increase buprenorphine prescribing for PCCs less likely to prescribe upon patient request. Future research is needed to develop interventions that increase PCC buprenorphine prescribing. Clinical trial registration: ClinicalTrials.gov. Identifier: NCT04198428. Clinical trial name: Clinical Decision Support for Opioid Use Disorders in Medical Settings (Compute 2.0).

Effectiveness of spinal manipulation and biopsychosocial self-management compared to medical care for low back pain: a randomized trial study protocol

BACKGROUND

Chronic low back pain (cLBP) is widespread, costly, and burdensome to patients and health systems. Little is known about non-pharmacological treatments for the secondary prevention of cLBP. There is some evidence that treatments addressing psychosocial factors in higher risk patients are more effective than usual care. However, most clinical trials on acute and subacute LBP have evaluated interventions irrespective of prognosis.

METHODS

We have designed a phase 3 randomized trial with a 2 × 2 factorial design. The study is also a Hybrid type 1 trial with focus on intervention effectiveness while simultaneously considering plausible implementation strategies. Adults (n = 1000) with acute/subacute LBP at moderate to high risk of chronicity based on the STarT Back screening tool will be randomized in to 1 of 4 interventions lasting up to 8 weeks: supported self-management (SSM), spinal manipulation therapy (SMT), both SSM and SMT, or medical care. The primary objective is to assess intervention effectiveness; the secondary objective is to assess barriers and facilitators impacting future implementation. Primary effectiveness outcome measures are: (1) average pain intensity over 12 months post-randomization (pain, numerical rating scale); (2) average low back disability over 12 months post-randomization (Roland-Morris Disability Questionnaire); (3) prevention of cLBP that is impactful at 10-12 months follow-up (LBP impact from the PROMIS-29 Profile v2.0). Secondary outcomes include: recovery, PROMIS-29 Profile v2.0 measures to assess pain interference, physical function, anxiety, depression, fatigue, sleep disturbance, and ability to participate in social roles and activities. Other patient-reported measures include LBP frequency, medication use, healthcare utilization, productivity loss, STarT Back screening tool status, patient satisfaction, prevention of chronicity, adverse events, and dissemination measures. Objective measures include the Quebec Task Force Classification, Timed Up & Go Test, the Sit to Stand Test, and the Sock Test assessed by clinicians blinded to the patients' intervention assignment.

DISCUSSION

By targeting those subjects at higher risk this trial aims to fill an important gap in the scientific literature regarding the effectiveness of promising non-pharmacological treatments compared to medical care for the management of patients with an acute episode of LBP and the prevention of progression to a severe chronic back problem.

TRIAL REGISTRATION

ClinicalTrials.gov Identifier: NCT03581123.

Effectiveness of Spinal Manipulation and Biopsychosocial Self-Management compared to Medical Care for Low Back Pain: A Randomized Trial Study Protocol

Background Chronic low back pain (cLBP) is widespread, costly, and burdensome to patients and health systems. Little is known about non-pharmacological treatments for the secondary prevention of cLBP. There is some evidence that treatments addressing psychosocial factors in higher risk patients are more effective than usual care. However, most clinical trials on acute and subacute LBP have evaluated interventions irrespective of prognosis. Methods We have designed a phase 3 randomized trial with a 2x2 factorial design. The study is also a Hybrid type 1 trial with focus on intervention effectiveness while simultaneously considering plausible implementation strategies. Adults (n = 1000) with acute/subacute LBP at moderate to high risk of chronicity based on the STarT Back screening tool will be randomized in to 1 of 4 interventions lasting up to 8 weeks: supported self-management (SSM), spinal manipulation therapy (SMT), both SSM and SMT, or medical care. The primary objective is to assess intervention effectiveness; the secondary objective is to assess barriers and facilitators impacting future implementation. Primary effectiveness outcome measures are: (1) average pain intensity over 12 months post-randomization (pain, numerical rating scale); (2) average low back disability over 12 months post-randomization (Roland-Morris Disability Questionnaire); (3) prevention of cLBP that is impactful at 10-12 months follow-up (LBP impact from the PROMIS-29 Profile v2.0). Secondary outcomes include: recovery, PROMIS-29 Profile v2.0 measures to assess pain interference, physical function, anxiety, depression, fatigue, sleep disturbance, and ability to participate in social roles and activities. Other patient-reported measures include LBP frequency, medication use, healthcare utilization, productivity loss, STarT Back screening tool status, patient satisfaction, prevention of chronicity, adverse events, and dissemination measures. Objective measures include the Quebec Task Force Classification, Timed Up & Go Test, the Sit to Stand Test, and the Sock Test assessed by clinicians blinded to the patients' intervention assignment. Discussion By targeting those subjects at higher risk this trial aims to fill an important gap in the scientific literature regarding the effectiveness of promising non-pharmacological treatments compared to medical care for the management of patients with an acute episode of LBP and the prevention of progression to a severe chronic back problem. Trial registration: ClinicalTrials.gov Identifier: NCT03581123.

Integrating pragmatic and implementation science randomized clinical trial approaches: a PRagmatic Explanatory Continuum Indicator Summary-2 (PRECIS-2) analysis

BACKGROUND

Over the past two decades, pragmatic and implementation science clinical trial research methods have advanced substantially. Pragmatic and implementation studies have natural areas of overlap, particularly relating to the goal of using clinical trial data to leverage health care system policy changes. Few investigations have addressed pragmatic and implementation science randomized trial methods development while also considering policy impact.

METHODS

The investigation used the PRagmatic Explanatory Continuum Indicator Summary-2 (PRECIS-2) and PRECIS-2-Provider Strategies (PRECIS-2-PS) tools to evaluate the design of two multisite randomized clinical trials that targeted patient-level effectiveness outcomes, provider-level practice changes and health care system policy. Seven raters received PRECIS-2 training and applied the tools in the coding of the two trials. Descriptive statistics were produced for both trials, and PRECIS-2 wheel diagrams were constructed. Interrater agreement was assessed with the Intraclass Correlation (ICC) and Kappa statistics. The Rapid Assessment Procedure Informed Clinical Ethnography (RAPICE) qualitative approach was applied to understanding integrative themes derived from the PRECIS-2 ratings and an end-of-study policy summit.

RESULTS

The ICCs for the composite ratings across the patient and provider-focused PRECIS-2 domains ranged from 0.77 to 0.87, and the Kappa values ranged from 0.25 to 0.37, reflecting overall fair-to-good interrater agreement for both trials. All four PRECIS-2 wheels were rated more pragmatic than explanatory, with composite mean and median scores ≥ 4. Across trials, the primary intent-to-treat analysis domain was consistently rated most pragmatic (mean = 5.0, SD = 0), while the follow-up/data collection domain was rated most explanatory (mean range = 3.14-3.43, SD range = 0.49-0.69). RAPICE field notes identified themes related to potential PRECIS-2 training improvements, as well as policy themes related to using trial data to inform US trauma care system practice change; the policy themes were not captured by the PRECIS-2 ratings.

CONCLUSIONS

The investigation documents that the PRECIS-2 and PRECIS-2-PS can be simultaneously used to feasibly and reliably characterize clinical trials with patient and provider-level targets. The integration of pragmatic and implementation science clinical trial research methods can be furthered by using common metrics such as the PRECIS-2 and PRECIS-2-PS. Future study could focus on clinical trial policy research methods development.

TRIAL REGISTRATION

DO-SBIS ClinicalTrials.gov NCT00607620. registered on January 29, 2008. TSOS ClinicalTrials.gov NCT02655354, registered on July 27, 2015.

Evaluation of an inpatient occupation-based group program using a process evaluation framework

INTRODUCTION

Occupation-based groups are used in inpatient rehabilitation to enhance delivery and intensity of intervention; however, little research is available to understand their effectiveness. The aim of this study was to examine the process of an occupation-based group to understand mechanisms and success factors.

METHODS

A mixed methods process evaluation using an evidence-based framework guided implementation and analysis. Participants were those involved in the 'LifeSkills' group run daily in an adult inpatient general rehabilitation ward. Quantitative administrative data, goal achievement outcome measures, group observations and qualitative semistructured interviews were conducted.

RESULTS

Thirty participants were recruited. Factors for success included consistency in group structure and support, using meaningful practise opportunities and facilitating a real-world experience. There was no significant relationship between patient outcomes and dose of training or patient demographics.

CONCLUSION

This evaluation contributes to a growing body of evidence for incorporating occupation-based approaches into rehabilitation and offers insights into practice implementation.

Using the RE-AIM framework to evaluate the implementation of scaling-up the Friendship Bench in Zimbabwe - a quantitative observational study

BACKGROUND

This study aimed to evaluate the real-world implementation of the Friendship Bench (FB) - an evidence-based brief psychological intervention delivered by community health workers (CHWs) - three years after its implementation in three city health departments in Zimbabwe. Implementation sites were evaluated according to their current performance using the RE-AIM framework making this one of the first evaluations of a scaled-up evidence-based psychological intervention in sub-Saharan Africa (SSA).

METHODS

Using the RE-AIM guide ( www.re-aim.org ), the authors designed quantitative indicators based on existing FB implementation data. Thirty-six primary health care clinics (PHC) in Harare (n=28), Chitungwiza (n=4) and Gweru (n=4) were included. Among these clinics 20 were large comprehensive health care centers, 7 medium (mostly maternal and child healthcare) and 9 small clinics (basic medical care and acting as referral clinic). Existing data from these clinics, added to additionally collected data through interviews and field observations were used to investigate and compare the performance of the FB across clinics. The focus was on the RE-AIM domains of Reach, Adoption, and Implementation.

RESULTS

Small clinics achieved 34% reach, compared to large (15%) and medium clinics (9%). Adoption was high in all clinic types, ranging from 59% to 71%. Small clinics led the implementation domain with 53%, followed by medium sized clinics 43% and large clinics 40%. Small clinics performed better in all indicators and differences in performance between small and large clinics were significant. Program activity and data quality depends on ongoing support for delivering agents and buy-in from health authorities.

CONCLUSION

The Friendship Bench program was implemented over three years transitioning from a research-based implementation program to one led locally. The Reach domain showed the largest gap across clinics where larger clinics performed poorly relative to smaller clinics and should be a target for future implementation improvements. Program data needs to be integrated into existing health information systems. Future studies should seek to optimize scale-up and sustainment strategies to maintain effective task-shared psychological interventions in SSA.

Assessing the Pragmatic Nature of mHealth Interventions Promoting Physical Activity: A Systematic Review and Meta-Analysis (Preprint)

BACKGROUND

Mobile health (mHealth) apps can promote physical activity; however, the pragmatic nature (ie, how well research translates into real-world settings) of these studies is unknown. The impact of study design choices, for example, intervention duration, on intervention effect sizes is also understudied.

OBJECTIVE

This review and meta-analysis aims to describe the pragmatic nature of recent mHealth interventions for promoting physical activity and examine the associations between study effect size and pragmatic study design choices.

METHODS

The PubMed, Scopus, Web of Science, and PsycINFO databases were searched until April 2020. Studies were eligible if they incorporated apps as the primary intervention, were conducted in health promotion or preventive care settings, included a device-based physical activity outcome, and used randomized study designs. Studies were assessed using the Reach, Effectiveness, Adoption, Implementation, Maintenance (RE-AIM) and Pragmatic-Explanatory Continuum Indicator Summary-2 (PRECIS-2) frameworks. Study effect sizes were summarized using random effect models, and meta-regression was used to examine treatment effect heterogeneity by study characteristics.

RESULTS

Overall, 3555 participants were included across 22 interventions, with sample sizes ranging from 27 to 833 (mean 161.6, SD 193.9, median 93) participants. The study populations' mean age ranged from 10.6 to 61.5 (mean 39.6, SD 6.5) years, and the proportion of males included across all studies was 42.8% (1521/3555). Additionally, intervention lengths varied from 2 weeks to 6 months (mean 60.9, SD 34.9 days). The primary app- or device-based physical activity outcome differed among interventions: most interventions (17/22, 77%) used activity monitors or fitness trackers, whereas the rest (5/22, 23%) used app-based accelerometry measures. Data reporting across the RE-AIM framework was low (5.64/31, 18%) and varied within specific dimensions (Reach=44%; Effectiveness=52%; Adoption=3%; Implementation=10%; Maintenance=12.4%). PRECIS-2 results indicated that most study designs (14/22, 63%) were equally explanatory and pragmatic, with an overall PRECIS-2 score across all interventions of 2.93/5 (SD 0.54). The most pragmatic dimension was flexibility (adherence), with an average score of 3.73 (SD 0.92), whereas follow-up, organization, and flexibility (delivery) appeared more explanatory with means of 2.18 (SD 0.75), 2.36 (SD 1.07), and 2.41 (SD 0.72), respectively. An overall positive treatment effect was observed (Cohen d=0.29, 95% CI 0.13-0.46). Meta-regression analyses revealed that more pragmatic studies (-0.81, 95% CI -1.36 to -0.25) were associated with smaller increases in physical activity. Treatment effect sizes were homogenous across study duration, participants' age and gender, and RE-AIM scores.

CONCLUSIONS

App-based mHealth physical activity studies continue to underreport several key study characteristics and have limited pragmatic use and generalizability. In addition, more pragmatic interventions observe smaller treatment effects, whereas study duration appears to be unrelated to the effect size. Future app-based studies should more comprehensively report real-world applicability, and more pragmatic approaches are needed for maximal population health impacts.

TRIAL REGISTRATION

PROSPERO CRD42020169102; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=169102.

Implementation Science: Increasing the Public Health Impact of Audiology Research

PURPOSE

Implementation science aims to facilitate the use of evidence-based programs, practices, and policies in routine care settings. In audiology, as in other health disciplines, there is a persistent research-to-practice gap. Improving the adoption, reach, implementation, and sustainment of effective interventions in audiology would increase their public health impact, ensuring that all individuals needing hearing health care services could benefit from innovations and evidence-based best practices. This tutorial provides an introductory overview of implementation science relevant to the field of audiology, including Internet-based practices and interventions.

METHOD

Major concepts and themes of implementation science are presented, including implementation outcomes, implementation science frameworks, implementation strategies, current topics in implementation science, and study design considerations. Recent publications in audiology are highlighted to illustrate implementation science concepts and themes. The relevance of each topic to the use of evidence-based programs, practices, and policies in audiology is highlighted with reference to recent research in the field.

CONCLUSIONS

Challenges in the widespread delivery of evidence-based audiological practices and interventions limit their public health impact. The application of implementation science principles and methods in audiology research, as demonstrated in other areas of health research, can increase our focus on ensuring that effective practices are widely available, accessible, equitable, and sustainable to improve the lives of those who need them.

Methods to improve the translation of evidence-based interventions: A primer on dissemination and implementation science for occupational safety and health researchers and practitioners

Objective

A limited focus on dissemination and implementation (D&I) science has hindered the uptake of evidence-based interventions (EBIs) that reduce workplace morbidity and mortality. D&I science methods can be used in the occupational safety and health (OSH) field to advance the adoption, implementation, and sustainment of EBIs for complex workplaces. These approaches should be responsive to contextual factors, including the needs of partners and beneficiaries (such as employers, employees, and intermediaries).

Methods

By synthesizing seminal literature and texts and leveraging our collective knowledge as D&I science and/or OSH researchers, we developed a D&I science primer for OSH. First, we provide an overview of common D&I terminology and concepts. Second, we describe several key and evolving issues in D&I science: balancing adaptation with intervention fidelity and specifying implementation outcomes and strategies. Next, we review D&I theories, models, and frameworks and offer examples for applying these to OSH research. We also discuss widely used D&I research designs, methods, and measures. Finally, we discuss future directions for D&I science application to OSH and provide resources for further exploration.

Results

We compiled a D&I science primer for OSH appropriate for practitioners and evaluators, especially those newer to the field.

Conclusion

This article fills a gap in the OSH research by providing an overview of D&I science to enhance understanding of key concepts, issues, models, designs, methods and measures for the translation into practice of effective OSH interventions to advance the safety, health and well-being of workers.

A-REST (Activity to Reduce Excessive Sitting Time): A Feasibility Trial to Reduce Prolonged Sitting in Police Staff

The aim of this study was to evaluate the acceptability and feasibility of a theory-derived sedentary workplace intervention for police office staff. Twenty-four staff participated in an 8-week intervention (single arm, pre-post design) incorporating an education session, team competition with quick response (QR) codes, team trophy, weekly leaderboard newsletters, a self-monitoring phone app, and electronic prompt tools. The intervention supported participants to reduce and break up their sitting time with three minutes of incidental movement every 30 min at work. Feasibility and acceptability were assessed using mixed methods via the RE-AIM QuEST and PRECIS-2 frameworks. The intervention was highly pragmatic in terms of eligibility, organisation, adherence, outcome, and analysis. It was slightly less pragmatic on recruitment and setting. Delivery and follow-up were more explanatory. Reach and adoption indicators demonstrated feasibility among police staff, across a range of departments, who were demographically similar to participants in previous office-based multi-component interventions. The intervention was delivered mostly as planned with minor deviations from protocol (implementation fidelity). Participants perceived the intervention components as highly acceptable. Results showed improvements in workplace sitting and standing, as well as small improvements in weight and positive affect. Evaluation of the intervention in a fully powered randomised controlled trial to assess behaviour and health outcomes is recommended.

Protocol for a process evaluation: face-to-face physiotherapy compared with a supported home exercise programme for the management of musculoskeletal conditions: the REFORM trial

INTRODUCTION

The REFORM (REhabilitation FOR Musculoskeletal conditions) trial is a non-inferiority randomised controlled trial (n=210) designed to determine whether a supported home exercise programme is as good or better than a course of face-to-face physiotherapy for the management of some musculoskeletal conditions. The trial is currently being conducted across Sydney government hospitals in Australia. This process evaluation will run alongside the REFORM trial. It combines qualitative and quantitative data to help explain the trial results and determine the feasibility of rolling out supported home exercise programmes in settings similar to the REFORM trial.

METHODS AND ANALYSIS

Two theoretical frameworks underpin our process evaluation methodology: the Realist framework (context, mechanism, outcomes) considers the causal assumptions as to why a supported home exercise programme may be as good or better than face-to-face physiotherapy in terms of the context, mechanisms and outcomes of the trial. The RE-AIM framework describes the Reach, Effectiveness, Adoption, Implementation and Maintenance of the intervention. These two frameworks will be broadly used to guide this process evaluation using a mixed-methods approach. For example, qualitative data will be derived from interviews with patients, healthcare professionals and stakeholders, and quantitative data will be collected to determine the cost and feasibility of providing supported home exercise programmes. These data will be analysed iteratively before the analysis of the trial results and will be triangulated with the results of the primary and secondary outcomes.

ETHICS AND DISSEMINATION

This trial will be conducted in accordance with the National Health and Medical Research Council National Statement on Ethical Conduct in Human Research (2018) and the Note for Good Clinical Practice (CPMP/ICH-135/95). Ethical approval was obtained on 17 March 2017 from the Northern Sydney Local Health District Human Research Ethics Committee (trial number: HREC/16HAWKE/431-RESP/16/287) with an amendment for the process evaluation approved on 4 February 2020. The results of the process evaluation will be disseminated through publications in peer-reviewed journals and presentations at scientific conferences.

TRIAL REGISTRATION NUMBER

ACTRN12619000065190.

Systematic Review of the “Pragmatism” of Pragmatic Critical Care Trials

To assess the pragmatism of published critical care randomized controlled trials self-described as pragmatic using a validated tool.

Feasibility of Implementing Cancer-Specific Community-Based Exercise Programming: A Multi-Centre Randomized Trial

BACKGROUND

There is growing recognition of the importance of reporting preliminary work on the feasibility of a trial. The present study aimed to assess the feasibility of (1) a proposed fitness testing battery, and (2) processes related to the implementation of cancer-specific exercise programming in a community setting.

METHODS/DESIGN

A randomized controlled implementation feasibility trial was performed in advance of a large-scale implementation study. Eligible participants within 18 months of a cancer diagnosis were randomized to immediate or delayed community-based exercise at YMCA locations in Calgary and Edmonton, Canada for an 8-week period. The primary outcome for the trial was the feasibility of the physical fitness testing battery, defined as a 70% or greater completion rate across the 24-week study period. The Reach, Effectiveness, Adoption, Implementation and Maintenance (RE-AIM) framework was used to evaluate processes related to implementation of the exercise program across the two sites.

RESULTS

Eighty participants were recruited, 73 (91%) completed the 8-week trial, and 68 (85%) completed the 16- and 24-week follow-ups. Sixty participants (75%) completed the full physical fitness test battery at each time point, and 59 (74%) completed the patient-reported outcome measures. Statistically significant between-group differences were found in favor of the exercise group for functional aerobic capacity, upper and lower extremity strength, and symptoms. Differences were found between the sites, however, in completion rates and processes related to program implementation.

DISCUSSION

Findings suggest the need for minor adaptations to the physical fitness battery and outcome measures to better fit the community context. While findings support feasibility, context-specific challenges related to implementation processes were identified.

Anticipatory care planning for community-dwelling older adults at risk of functional decline: a feasibility cluster randomized controlled trial

Objectives

To determine the feasibility, implementation and outcomes of an Anticipatory Care Planning (ACP) intervention in primary care to assist older adults at risk of functional decline by developing a personalized support plan.

Design

Feasibility cluster randomized control trial.

Setting and participants

Eight primary care practices (four in Northern Ireland, United Kingdom and four in the Republic of Ireland) were randomly assigned to either intervention or control arm. Eligible patients were those identified in each practice as 70 years of age or older and assessed as at risk of functional decline. Study participants (intervention n = 34, control n = 31) and research staff were not blinded to group assignment.

Anticipatory care intervention

The intervention delivered by a registered nurse including: a) a home-based patient assessment; b) care planning on the basis of a holistic patient assessment, and c) documentation of a support plan.

Outcome measures

A conceptual framework (RE-AIM) guided the assessment on the potential impact of the ACP intervention on patient quality of life, mental health, healthcare utilisation, costs, perception of person-centred care, and reduction of potentially inappropriate prescribing. Data were collected at baseline and at 10 weeks and six months following delivery of the intervention.

Results

All pre-specified feasibility indicators were met. Patients were unanimous in the acceptance of the ACP intervention. Health care providers viewed the ACP intervention as feasible to implement in routine clinical practice with attending community supports. While there were no significant differences on the primary outcomes (EQ-5D-5L: -0.07 (-0.17, 0.04) p = .180; CES-D: 1.2 (-2.5, 4.8) p = .468) and most secondary measures, ancillary analysis on social support showed responsiveness to the intervention. Incremental cost analysis revealed a mean reduction in costs of €320 per patient (95% CI -31 to 25; p = 0.82) for intervention relative to the control.

Conclusions

We successfully tested the ACP intervention in primary care settings and have shown that it is feasible to implement. The ACP intervention deserves further testing in a definitive trial to determine whether its implementation would lead to better outcomes or reduced costs.

Trial registration

Clinicaltrials.gov, ID: NCT03902743. Registered on 4 April 2019.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12877-022-03128-x.

In It for the Long Haul: RE-AIM Evaluation of a Preschool Programme Implementing and Maintaining Adult-Initiated Motor Skill Development and Physical Activity across a Two-Year Period

Good motor skills (MS) are considered important for children's social, psychological and physical development and general physical activity (PA) levels. The Motor skill in Preschool study (MiPS) aimed to optimize children's MS through weekly PA sessions. The aim of this study is to use the RE-AIM framework to report the two-year implementation process of MiPS since the programme's initiation. Data were collected through a staff questionnaire based on the RE-AIM framework. Data were collected at three months, one year and two years after initiation. Results show that the pedagogical staff believes that the programme promotes MS in children. Implementation measures only showed medium to low fidelity concerning the core element of performing adult-initiated PA sessions with a duration of at least 45 min 4 days a week. The largest barrier was finding the time to plan these PA sessions. Still, the content of the PA sessions achieved high fidelity scores and the programme was deemed suitable for staff's everyday practice and in alignment with the stated pedagogical goals. The mandatory competence development course was highly valued as strong implementation support. It is notable that there is a large variation in the implementation among the preschools with some struggling more than others.

Evaluation of a Healthy Relationship Smartphone App With Indigenous Young People: Protocol for a Co-designed Stepped Wedge Randomized Trial

BACKGROUND

We co-designed a smartphone app, Harmonised, with taitamariki (young people aged 13-17 years) to promote healthy intimate partner relationships. The app also provides a pathway for friends and family, or whānau (indigenous Māori extended family networks), to learn how to offer better support to taitamariki.

OBJECTIVE

The aim of our taitamariki- and Māori-centered study is to evaluate the implementation of the app in secondary schools. The study tests the effectiveness of the app in promoting taitamariki partner relationship self-efficacy (primary outcome).

METHODS

We co-designed a pragmatic, randomized, stepped wedge trial (retrospectively registered on September 12, 2019) for 8 Aotearoa, New Zealand, secondary schools (years 9 through 13). The schools were randomly assigned to implement the app in 1 of the 2 school terms. A well-established evaluation framework (RE-AIM [Reach, Effectiveness, Adoption, Implementation, Maintenance]) guided the selection of mixed data collection methods. Our target sample size is 600 taitamariki enrolled across the 8 schools. Taitamariki will participate by completing 5 web-based surveys over a 15-month trial period. Taitamariki partner relationship self-efficacy (primary outcome) and well-being, general health, cybersafety management, and connectedness (secondary outcomes) will be assessed with each survey. The general effectiveness hypotheses will be tested by using a linear mixed model with nested participant, year-group, and school random effects. The primary analysis will also include testing effectiveness in the Māori subgroup.

RESULTS

The study was funded by the New Zealand Ministry of Business, Innovation, and Employment in October 2015 and approved by the Auckland University of Technology Ethics Committee on May 3, 2017 (application number: 17/71).

CONCLUSIONS

This study will generate robust evidence evaluating the impact of introducing a healthy relationship app in secondary schools on taitamariki partner relationship self-efficacy, well-being, general health, cybersafety management, and connectedness. This taitamariki- and indigenous Māori-centered research fills an important gap in developing and testing strengths-based mobile health interventions in secondary schools.

TRIAL REGISTRATION

Australian New Zealand Clinical Trials Registry ACTRN12619001262190; https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=377584.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR1-10.2196/24792.

The Role of Local Leaders in the Implementation of Adult-Initiated Motor Skill Development and Physical Activity in Preschool-A Case Study

Good motor skills (MS) and physical activity (PA) are considered important for children’s physical, social, and psychological development. The Motor skills in PreSchool (MiPS) study-Denmark aimed to optimize children’s MS through weekly PA sessions. The aim of this paper is to explore the role of local preschool leaders and their strategies in influencing the implementation of MiPS into daily practice. Leaders from all seven preschools in the project were interviewed. The results show that preschool leaders used communication (setting an agenda and dedicating speaking time to address the program at staff meetings) and reflective questioning about the pedagogic staff’s current practice in relation to the program (adding focus on MS and PA) as their main course of action. Through this form of communication and reflective questioning, the preschool leaders aimed to involve the staff and heighten their sensemaking in the existing practice while also ensuring that the implementation of the program was kept in progress. In sum, future implementation of an MS and PA initiative in preschools should put emphasis on a shared responsibility between leaders and staff combined with an adaptive approach in which the existing practice is reshaped rather than just increasing the workload of the pedagogic staff.

Protocol refinement for a diabetes pragmatic trial using the PRECIS-2 framework

Background

This report describes how we refined a protocol for a pragmatic comparative effectiveness study of two models of an evidence-based diabetes shared medical appointment intervention and used the PRECIS-2 rating system to evaluate these adaptations.

Methods

We report primary data collected between June and August 2019, and protocol refinements completed between 2018 and 2020. Twenty-two members of the study team collaborated in protocol refinement and completed the PRECIS-2 ratings of study pragmatism. We discuss study design refinements made to achieve the desired level of pragmatism vs. experimental control for each of the nine PRECIS-2 dimensions. Study team members received training on PRECIS-2 scoring and were asked to rate the study protocol on the nine PRECIS-2 dimensions. Ratings were compared using descriptive statistics.

Results

In general, the PRECIS-2 ratings revealed high levels of pragmatism, but somewhat less pragmatic ratings on the categories of Delivery and Organization (costs and resources). This variation was purposeful, and we provide the rationale for and steps taken to obtain the targeted level of pragmatism on each PRECIS-2 dimension, as well as detail design changes made to a) make the design more pragmatic and b) address COVID-19 issues. There was general agreement among team members and across different types of stakeholders on PRECIS-2 ratings.

Conclusions

We discuss lessons learned from use of PRECIS-2 and experiences in refining the study to be maximally pragmatic on some dimensions and less so on other dimensions. This paper expands on prior research by describing actions to achieve higher levels of pragmatism and revise our protocol fit to the changed context. We make recommendations for future use of PRECIS-2 to help address changing context and other strategies for the planning of and transparent reporting on pragmatic research and comparative effectiveness research.

Trial registration

Clinicaltrials.gov Registration ID: NCT03590041.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-021-07084-x.

Encouraging and evaluating limit-setting among on-line gamblers: a naturalistic randomized controlled trial

AIMS

We tested the effectiveness of three different messages designed to increase limit-setting on gambling sites and sent these via e-mail or in-account notification to compare delivery modes. As a secondary aim, we examined the effects of limit-setting on gambling behaviour.

DESIGN

A pre-registered, naturalistic randomized control trial using a 3 × 2 plus control design.

SETTING

Four on-line Australian sports and racing wagering websites.

PARTICIPANTS

A total of 31 989 wagering customers (reduced to 26,560 after eligibility screening) who had placed bets on at least 5 days in the past 30 [mean age = 41.4, standard deviation (SD) = 14.3; 79% male].

INTERVENTIONS AND COMPARATORS

Messages were sent via e-mail or in-account notification by on-line gambling operators and were designed to either: (1) be informative, describing the availability and purpose of the tool (informative messages), (2) highlight the benefits other people receive from using the tool (social messages) or (3) promote the benefit individuals could receive from using the tool (personal messages). A control group who did not receive messages was monitored for comparison.

MEASUREMENTS

Our primary outcome was the number of customers who set a deposit limit within 5 days of receiving messages and secondary outcomes included pre- and post-message betting behaviour (e.g. average daily wager).

FINDINGS

One hundred and sixty-one (0.71%) customers sent messages set limits compared to three (0.08%) controls [adjusted odds ratio (aOR) = 8.17, 95% confidence interval (CI) = 2.99, 33.76)]. Social and personal messages were no more effective than informative messages (aOR = 0.98, 95% CI = 0.65, 1.48; aOR = 0.93, 95% CI = 0.60, 1.44) and in-account messages were no more effective than e-mails (aOR = 1.02, 95% CI = 0.71, 1.49). Customers who set limits significantly decreased their average daily wager, the SD of daily wager, net loss and betting intensity compared with non-limit-setters.

CONCLUSIONS

Messages to on-line gambling website customers are inexpensive, and may lead to small but impactful increases in setting deposit limits. Limit-setting may be an effective strategy for reducing gambling expenditure and intensity.

Pragmatic Evaluation of Older Adults' Physical Activity in Scale-Up Studies: Is the Single-Item Measure a Reasonable Option?

Convergent validity and responsiveness to change of the single-item physical activity measure were assessed in adults aged 60 years and older, at baseline (n = 205) and 6 months (n = 177) of a health promotion program, Choose to Move. Spearman correlations were used to examine associations between physical activity as measured by the single-item measure and the Community Health Activities Model Program for Seniors (CHAMPS) questionnaire at baseline and for 6-month change in all participants and for sex and age (60-74 years, and ≥75 years) subgroups. Effect size assessed responsiveness to change in physical activity for both tools. Baseline physical activity by the single-item measure correlated moderately with physical activity by the CHAMPS questionnaire in all participants and subgroups. Correlations were weaker for change in physical activity. Effect size for physical activity change was larger for the single-item measure than for the CHAMPS questionnaire. The single-item measure is a valid, pragmatic tool for use in intervention and scale-up studies with older adults.

A mixed methods study to evaluate participatory mapping for rural water safety planning in western Kenya

Water safety planning is an approach to ensure safe drinking-water access through comprehensive risk assessment and water supply management from catchment to consumer. However, its uptake remains low in rural areas. Participatory mapping, the process of map creation for resource management by local communities, has yet to be used for rural water safety planning. In this mixed methods study, to evaluate the validity of participatory mapping outputs for rural water safety planning and assess community understanding of water safety, 140 community members in Siaya County, Kenya, attended ten village-level participatory mapping sessions. They mapped drinking-water sources, ranked their safety and mapped potential contamination hazards. Findings were triangulated against a questionnaire survey of 234 households, conducted in parallel. In contrast to source type ranking for international monitoring, workshop participants ranked rainwater's safety above piped water and identified source types such as broken pipes not explicitly recorded in water source typologies often used for formal monitoring. Participatory mapping also highlighted the overlap between livestock grazing areas and household water sources. These findings were corroborated by the household survey and subsequent participatory meetings. However, comparison with household survey data suggested participatory mapping outputs omitted some water sources and landscape-scale contamination hazards, such as open defecation areas or flood-prone areas. In follow-up visits, participant groups ranked remediation of rainwater harvesting systems as the most acceptable intervention to address hazards. We conclude that participatory mapping can complement other established approaches to rural water safety planning by capturing informally managed source use and facilitating community engagement.

Resources, methods, and data infrastructure to promote research in dementia care, caregiving, and services

BACKGROUND

The National Institute on Aging, in conjunction with the Department of Health and Human Services as part of the National Alzheimer's Project Act, hosted a 2020 Dementia Care, Caregiving, and Services Research Summit Virtual Meeting Series on August 13, 2020. This article reflects three presentations related to Theme 6: Research Resources, Methods, and Data Infrastructure. Dr. Bynum discussed the challenges of identifying people for population- and healthcare-based research, including how definitions of dementia have changed over time, the opportunities and challenges inherent in the use of electronic data sources, and the need to fit data collection strategies to research goals and questions. Dr. Travison provided an overview on the growing use of embedded pragmatic clinical trials (ePCTs) and how to enhance their impact in dementia research. Dr. Wendler presented on the ethical considerations relevant to consent for dementia research, including assessment of decisional capacity and the role of decisional surrogates.

CONCLUSIONS

The availability of claims data, electronic health records, and other sources of "existing" data has made the use and development of ePCTs both easier and more appealing. Among other things, they offer advantages in terms of lower cost and generalizability to real-world settings. This is turn has necessitated the use of informatic and analytic approaches to account for some of the limitations and complexities of such data, including multilevel clustering and the need to link and jointly analyze data from the person with dementia and those of their care partner. As part of this process, it will be important to broaden the scope of who is assessed for decisional capacity, make those assessments more study specific, and assist surrogates in making decisions based on what the individual would have chosen for themselves if capacitated (i.e., substituted judgment).

Evaluating implementation and pragmatism of cancer-specific exercise programs: a scoping review

PURPOSE

Studies in exercise oncology are increasingly evaluating the implementation of cancer-specific exercise programming in real-world settings. Evaluating current implementation efforts and identifying current research gaps may inform future implementation. This scoping review explores studies implementing cancer-specific exercise programs to determine their pragmatic characteristics and evaluate potential for translation into practice.

METHODS

A systematic literature search was conducted using five databases (up to July 2020) to identify implementation studies that used qualitative or quantitative methods. Program pragmatism was evaluated using the PRECIS/RE-AIM tool.

RESULTS

Fourteen articles were included for review. While programs were moderately pragmatic (average: 3.4; range: 2.8-4.2), variability was observed between scoring domains. Programs scored higher (more pragmatic) in domains typical for effectiveness studies, including participant eligibility, follow-up intensity, primary trial outcome, and analysis of primary outcome. In contrast, programs scored lower (less pragmatic) in domains aligned with implementation science, including intervention flexibility, adoption, implementation, and maintenance. Limited information was reported regarding program fidelity, adaptations, and maintenance/sustainability.

CONCLUSION

Researchers should consider the pragmatism of interventions, the nuances of program adoption and implementation at the setting level, and the transition and integration of programming into the healthcare system. Future studies may benefit from the inclusion of decision-makers and implementation experts and shifting focus towards flexible programming. PRECIS/RE-AIM may facilitate the evaluation of programs throughout the study design and implementation process.

IMPLICATIONS FOR CANCER SURVIVORS

The implementation of pragmatic exercise programs that are both scalable and sustainable is needed so that cancer survivors can experience exercise-related benefits beyond research settings.

Effectiveness and Reach of the Primary Palliative Care for Emergency Medicine (PRIM-ER) Pilot Study: a Qualitative Analysis

BACKGROUND

Palliative care interventions in the ED capture high-risk patients at a time of crisis and can dramatically improve patient-centered outcomes.

OBJECTIVE

To understand the facilitators that contributed to the success of the Primary Palliative Care for Emergency Medicine (PRIM-ER) quality improvement pilot intervention.

DESIGN

Effectiveness was evaluated through semi-structured interviews. Reach outcomes were measured by percent of all full-time emergency providers (physicians, physician assistants, nurses) who completed the intervention education components and baseline survey assessing attitudes and knowledge on end-of-life care.

PARTICIPANTS

Emergency medicine providers affiliated with two medical centers (N = 197). Interviews conducted with six key informants at both institutions.

APPROACH

Interviews were recorded, transcribed, and analyzed using deductive and inductive approaches. Descriptive statistics include reach outcomes and baseline survey results.

KEY RESULTS

Both sites successfully implemented all components of the intervention and achieved a high level (> 75%) of intervention reach. Two themes emerged as facilitators to successful effectiveness facilitators of PRIM-ER: (1) institutional leadership support and (2) leveraging established quality improvement (QI) processes. Institutional support included leveraging leadership with authority to (a) mandate trainings; (b) substitute PRIM-ER education for normally scheduled education; and (c) provide protected time to implement intervention components. Effectiveness was also enhanced by capitalizing on existing QI processes which included (a) leveraging interdisciplinary partnerships and communication plans and (b) monitoring performance improvement data.

CONCLUSIONS

Capitalizing on strong institutional leadership support and established QI processes enhanced the reach and effectiveness of the PRIM-ER pilot. These findings will guide the PRIM-ER researchers in scaling up the intervention in the remaining 33 sites, as well as enhance the planning of other complex quality improvement interventions in clinical settings.

REGISTRATION DETAILS

ClinicalTrials.gov Identifier: NCT03424109; Grant Number: AT009844-01.

Real-world experiences with generating real-world evidence: Case Studies from PCORI's pragmatic clinical Studies program

BACKGROUND

Over the last decade, randomized studies evaluating outcomes of health care interventions conducted in real-world settings-often termed "pragmatic trials"-have come to be seen as an important means of obtaining relevant, actionable evidence to guide health care decisions. Despite extensive writing on methodological considerations in pragmatic trial design, limited information exists regarding the practical and logistical challenges encountered in carrying out rigorous randomized evaluations in highly representative, real-world contexts.

METHODS

The Patient Centered Outcomes Research Institute (PCORI) convened an expert panel in 2017 to examine common tradeoffs in study design and implementation through 3 "case studies" of in-progress, PCORI-funded pragmatic trials. This paper summarizes the findings of this panel, using the 3 examples to illustrate common implementation challenges encountered in pragmatic trials.

RESULTS

Investigators aimed to generate highly generalizable findings that could address real-world clinical decisions; however, practical considerations required that each study incorporate traditionally "explanatory" elements to achieve a "fit-for-purpose" approach to design and implementation. Within individual studies, efforts to balance pragmatic versus explanatory perspectives often involved multiple, diverse aspects of trial design and implementation, and the aspects of design and implementation where investigators reported encountering such tradeoffs varied across the three cases we examined.

CONCLUSIONS

Efforts to generate rigorous evidence that is generalizable to "real-world" practice require continuous and iterative efforts to balance "pragmatic" and "explanatory" perspectives. In each study examined, these tradeoffs were guided both by an overriding effort to maintain pragmatism and practical considerations that varied depending on the research question and study context.

The importance of decision intent within descriptions of pragmatic trials

OBJECTIVE

It is now more than 50 years since the concepts of explanatory and pragmatic attitudes toward trials were first discussed by Schwartz and Lellouch in their influential 1967 paper. Since then, there has been increasing focus on design aspects that may be consistent with more pragmatic attitudes within clinical trials, and a number of tools developed to assist investigators prospectively think about their trial design. Researchers have subsequently expressed interest in using these tools retrospectively to characterize trials as pragmatic or explanatory.

RESULTS

We suggest that recent attempts to retrospectively dichotomize trials solely on the basis of quantitative scoring of trial design features are flawed. Instead, we argue that there is a need to consider both the intent and design when assessing the degree of pragmatism within a trial.

CONCLUSION

The practical implication of our suggestion for trial reporting is that investigators should explicitly state the intent of the trial through a clear articulation of the decision that they hope will be informed by the trial results. This should be coupled with a completed PRagmatic-Explanatory Continuum Indicator Summary 2 assessment (or similar) with an explanation of study design choices to appropriately assess whether the study design is consistent with the study intent. We believe this will assist reviewers and knowledge users in making assessments of trials.

Recruitment and reach in a pragmatic behavioral weight loss randomized controlled trial: implications for real-world primary care practice

BACKGROUND

Obesity is a major risk factor behind some of the most common problems encountered in primary care. Although effective models for obesity treatment have been developed, the 'reach' of these interventions is poor and only a small fraction of primary care patients receive evidence-based treatment. The purpose of this study is to identify factors that impact the uptake (reach) of an evidence-based obesity treatment program within the context of a pragmatic cluster randomized controlled trial comparing three models of care delivery.

METHODS

Recruitment and reach were evaluated by the following measures: 1) mailing response rates, 2) referral sources among participants contacting the study team, 3) eligibility rates, 4) participation rates, and 5) representativeness based on demographics, co-morbid conditions, and healthcare utilization of 1432 enrolled participants compared to > 17,000 non-participants from the clinic-based patient populations. Referral sources and participation rates were compared across study arms and level of clinic engagement.

RESULTS

The response rate to clinic-based mailings was 13.2% and accounted for 66% of overall program recruitment. An additional 22% of recruitment came from direct clinic referrals and 11% from media, family, or friends. Of those screened, 87% were eligible; among those eligible, 86% enrolled in the trial. Participation rates did not vary across the three care delivery arms, but were higher at clinics with high compared to low provider involvement. In addition, clinics with high provider involvement had a higher rate of in clinic referrals (33% versus 16%) and a more representative sample with regards to BMI, rurality, and months since last clinic visit. However, across clinics, enrolled participants compared to non-participants were older, more likely to be female, more likely to have had a joint replacement but less likely to have CVD or smoke, and had fewer hospitalizations.

CONCLUSIONS

A combination of direct patient mailings and in-clinic referrals may enhance the reach of primary care behavioral weight loss interventions, although more proactive outreach is likely necessary for men, younger patients, and those at greater medial risk. Strategies are needed to enhance provider engagement in referring patients to behavioral weight loss programs.

TRIAL REGISTRATION

clnicialtrials.gov NCT02456636. Registered May 28, 2015, https://www.clinicaltrials.gov/ct2/results?cond=&term=RE-POWER&cntry=&state=&city=&dist=.

Anticipatory care planning intervention for older adults at risk of functional decline: study protocol for a primary care cluster feasibility randomised trial

BACKGROUND

The treatment and management of long-term health conditions is the greatest challenge facing health systems around the world today. Innovative approaches to patient care in the community such as Anticipatory Care Planning (ACP), which seek to help with the provision of high-quality comprehensive care to older adults at risk of functional decline, require evaluation. This study will evaluate one approach that will include primary care as the setting for ACP.

METHODS/DESIGN

This study will help to determine the feasibility for a definitive randomised trial to evaluate the implementation and outcomes of an ACP intervention. The intervention will be delivered by specially trained registered nurses in a primary care setting with older adults identified as at risk of functional decline. The intervention will comprise: (a) information collection via patient assessment; (b) facilitated informed dialogue between the patient, family carer, general practitioner and other healthcare practitioners; and, (c) documentation of the agreed support plan and follow-up review dates. Through a structured consultation with patients and their family carers, the nurses will complete a mutually agreed personalised support plan.

DISCUSSION

This study will determine the feasibility for a full trial protocol to evaluate the implementation and outcomes of an (ACP) intervention in primary care to assist older adults aged 70 years of age or older and assessed as being at risk of functional decline. The study will be implemented in two jurisdictions on the island of Ireland which employ different health systems but which face similar health challenges. This study will allow us to examine important issues, such as the impact of two different healthcare systems on the health of older people and the influence of different legislative interpretations on undertaking cross jurisdictional research in Ireland.

PROTOCOL VERSION

Version 1, 17 September 2019.

TRIAL REGISTRATION

Clinicaltrials.gov, ID: NCT03902743. Registered on 4 April 2019.

Protocol for the process and feasibility evaluations of a new model of primary care service delivery for managing pain and function in patients with knee osteoarthritis (PARTNER) using a mixed methods approach

INTRODUCTION

This protocol outlines the rationale, design and methods for the process and feasibility evaluations of the primary care management on knee pain and function in patients with knee osteoarthritis (PARTNER) study. PARTNER is a randomised controlled trial to evaluate a new model of service delivery (the PARTNER model) against 'usual care'. PARTNER is designed to encourage greater uptake of key evidence-based non-surgical treatments for knee osteoarthritis (OA) in primary care. The intervention supports general practitioners (GPs) to gain an understanding of the best management options available through online professional development. Their patients receive telephone advice and support for OA management by a centralised, multidisciplinary 'Care Support Team'. We will conduct concurrent process and feasibility evaluations to understand the implementation of this new complex health intervention, identify issues for consideration when interpreting the effectiveness outcomes and develop recommendations for future implementation, cost effectiveness and scalability.

METHODS AND ANALYSIS

The UK Medical Research Council Framework for undertaking a process evaluation of complex interventions and the Reach, Effectiveness, Adoption, Implementation and Maintenance (RE-AIM) frameworks inform the design of these evaluations. We use a mixed-methods approach including analysis of survey data, administrative records, consultation records and semistructured interviews with GPs and their enrolled patients. The analysis will examine fidelity and dose of the intervention, observations of trial setup and implementation and the quality of the care provided. We will also examine details of 'usual care'. The semistructured interviews will be analysed using thematic and content analysis to draw out themes around implementation and acceptability of the model.

ETHICS AND DISSEMINATION

The primary and substudy protocols have been approved by the Human Research Ethics Committee of The University of Sydney (2016/959 and 2019/503). Our findings will be disseminated to national and international partners and stakeholders, who will also assist with wider dissemination of our results across all levels of healthcare. Specific findings will be disseminated via peer-reviewed journals and conferences, and via training for healthcare professionals delivering OA management programmes. This evaluation is crucial to explaining the PARTNER study results, and will be used to determine the feasibility of rolling-out the intervention in an Australian healthcare context.

TRIAL REGISTRATION NUMBER

ACTRN12617001595303; Pre-results.

Should We Scale-Up? A Mixed Methods Process Evaluation of an Intervention Targeting Sedentary Office Workers Using the RE-AIM QuEST Framework

Background: Interventions targeting a reduction in sedentary behaviour in office workers need to be scaled-up to have impact. In this study, the RE-AIM QuEST framework was used to evaluate the potential for further implementation and scale-up of a consultation based workplace intervention which targeted both the reduction, and breaking up of sitting time. Methods: To evaluate the Springfield College sedentary behaviour intervention across multiple RE-AIM QuEST indicators; intervention participant, non-participant (employees who did not participate) and key informant (consultation delivery team; members of the research team and stakeholders in workplace health promotion) data were collected using interviews, focus groups and questionnaires. Questionnaires were summarized using descriptive statistics and interviews and focus groups were transcribed verbatim, and thematically analysed. Results: Barriers to scale-up were: participant burden of activity monitoring; lack of management support; influence of policy; flexibility (scheduling/locations); time and cost. Facilitators to scale up were: visible leadership; social and cultural changes in the workplace; high acceptability; existing health and wellbeing programmes; culture and philosophy of the participating college. Conclusions: There is potential for scale-up, however adaptations will need to be made to address the barriers to scale-up. Future interventions in office workers should evaluate for scalability during the pilot phases of research.

Early intervention services for non-psychotic mental health disorders: a scoping review protocol

INTRODUCTION

Worldwide mental health disorders are associated with a considerable amount of human suffering, disability and mortality. Yet, the provision of rapid evidence-based care to mitigate the human and economic costs of these disorders is limited. The greatest progress in developing and delivering early intervention services has occurred within psychosis. There is now growing support for and calls to extend such approaches to other diagnostic groups. The aim of this scoping review is to systematically map the emerging literature on early intervention services for non-psychotic mental health disorders, with a focus on outlining how services are structured, implemented and scaled.

METHODS AND ANALYSIS

The protocol was developed using the guidance for scoping reviews in the Joanna Briggs Institute manual and the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for scoping reviews checklist. A systematic search for published and unpublished literature will be conducted using the following databases: (1) MEDLINE, (2) PsycINFO, (3) HMIC, (4) EMBASE and (5) ProQuest. To be included, documents must describe and/or evaluate an early intervention service for adolescents or adults with a non-psychotic mental health disorder. There will be no restrictions on publication type, study design and date. Title and abstract, and full-text screening will be completed by one reviewer, with a proportion of articles screened in duplicate. Data analysis will primarily involve a qualitatively summary of the early intervention literature, the characteristics of early intervention services and key findings relating to their evaluation and implementation.

ETHICS AND DISSEMINATION

The synthesis of published and unpublished articles will not require ethical approval. The results of this scoping review will be published in a peer-reviewed journal and disseminated via social media, conference presentations and other knowledge translation activities.

The "pragmatic trial": An essentially contested concept?

For over 50 years, clinical research methodology has wrestled with the problem of the lack of correspondence between tests of treatments and applications of treatments. The former comprise of trials featuring scrupulous control of patient eligibility, treatment compliance, clinician expertise, follow-up intensity, and so on. In applying a validated treatment, the practitioner has to confront considerable real-world variation in potential patients and in implementation regimes. The remedy, going by the name of "pragmatic trials," is to conduct clinical trials in conditions corresponding more closely to everyday practice. This solution has proved easier to utter than to execute, and the paper reviews the extensive literature on pragmatic trials, seeking to assess whether it has terminated in clarity or contestation.

Improving Research Literacy in Diverse Minority Populations with a Novel Communication Tool

Racial/ethnic minorities are underrepresented in clinical research in the USA for multifarious reasons, including barriers to effective communication between researchers and potential research participants. To address the communication barriers between researchers and potential participants, we developed a Research Literacy Support (RLS) tool. The focus of this report is to present findings from the second and third phases of development that refined and assessed usability of the RLS tool. We utilized a mixed-methods approach that entailed iterative cognitive testing with participants (N = 52) from diverse racial/ethnic backgrounds and interviews with clinical research recruiters (N = 20) to modify and refine the design and content of the RLS tool (phase 2). This was followed by assessment of the usability of the RLS tool by 100 participants (phase 3). During phase 2, participants provided feedback about layout, word choice, and comprehension of the tool. In phase 3, participants recognized that they had gained knowledge about clinical research from the RLS tool, although they still had a substantial learning gap after using the tool, indicating an opportunity for further refinement. The RLS tool may help advance health equity by addressing communication barriers that may impede minority participation in clinical research.

Strengthening Geriatric Expertise in Swiss Nursing Homes: INTERCARE Implementation Study Protocol

OBJECTIVES

Nursing home (NH) residents with complex care needs ask for attentive monitoring of changes and appropriate in-house decision making. However, access to geriatric expertise is often limited with a lack of geriatricians, general practitioners, and/or nurses with advanced clinical skills, leading to potentially avoidable hospitalizations. This situation calls for the development, implementation, and evaluation of innovative, contextually adapted nurse-led care models that support NHs in improving their quality of care and reducing hospitalizations by investing in effective clinical leadership, geriatric expertise, and care coordination.

DESIGN

An effectiveness-implementation hybrid type 2 design to assess clinical outcomes of a nurse-led care model and a mixed-method approach to evaluate implementation outcomes will be applied. The model development, tailoring, and implementation are based on the Consolidated Framework for Implementation Research (CFIR).

SETTING

NHs in the German-speaking region of Switzerland.

PARTICIPANTS

Eleven NHs were recruited. The sample size was estimated assuming an average of .8 unplanned hospitalizations/1000 resident days and a reduction of 25% in NHs with the nurse-led care model.

INTERVENTION

The multilevel complex context-adapted intervention consists of six core elements (eg, specifically trained INTERCARE nurses or evidence-based tools like Identify, Situation, Background, Assessment and Recommendation [ISBAR]). Multilevel implementation strategies include leadership and INTERCARE nurse training and support.

MEASUREMENTS

The primary outcomes are unplanned hospitalizations/1000 care days. Secondary outcomes include unplanned emergency department visits, quality indicators (eg, physical restraint use), and costs. Implementation outcomes included, for example, fidelity to the model's core elements.

CONCLUSION

The INTERCARE study will provide evidence about the effectiveness of a nurse-led care model in the real-world setting and accompanying implementation strategies. J Am Geriatr Soc 67:2145-2150, 2019.

Walk This Way: Our Perspective on Challenges and Opportunities for Extension Statewide Walking Promotion Programs

Evidence-based walking programs exist and some have been tailored specifically for the national Cooperative Extension System; however, program outcomes and translational challenges and successes are underreported. This has presented a challenge to scaling the best-fit intervention for walking promotion within this national system. Here, we describe existing open-access walking programs as well as implications for improving the fit and sustainability of this intervention type within the system. Our experience provides suggestions for pragmatic data collection, infrastructure to support pragmatic data collection, and novel ways to disseminate best practices, as well as considerations for de-implementing what is not working.

RE-AIM Planning and Evaluation Framework: Adapting to New Science and Practice With a 20-Year Review

The RE-AIM planning and evaluation framework was conceptualized two decades ago. As one of the most frequently applied implementation frameworks, RE-AIM has now been cited in over 2,800 publications. This paper describes the application and evolution of RE-AIM as well as lessons learned from its use. RE-AIM has been applied most often in public health and health behavior change research, but increasingly in more diverse content areas and within clinical, community, and corporate settings. We discuss challenges of using RE-AIM while encouraging a more pragmatic use of key dimensions rather than comprehensive applications of all elements. Current foci of RE-AIM include increasing the emphasis on cost and adaptations to programs and expanding the use of qualitative methods to understand "how" and "why" results came about. The framework will continue to evolve to focus on contextual and explanatory factors related to RE-AIM outcomes, package RE-AIM for use by non-researchers, and integrate RE-AIM with other pragmatic and reporting frameworks.

Lessons from Outside and Within: Exploring Advancements in Methodology for Naturopathic Medicine Clinical Research

INTRODUCTION

Naturopathy is a mixture of both traditional and complementary medicine. It incorporates a broad set of health care practices that may or may not be traditional to that country or conventional medicine and are not fully integrated into the dominant health care system. Research required to evaluate or substantiate naturopathic medicine may not fall under the testing of randomized clinical trials, which opens up discussions on what is the best practice for research in naturopathic medicine.

DISCUSSION

Not only do advances in health research methodology offer important opportunities to progress naturopathic research, there are also areas where the unique characteristics of naturopathic philosophy and practice can impact other areas of health research. Some of the new advances in health research methodology involve whole-system research, pragmatic trials, template for intervention description and replication protocols for complex interventions, patient-centered care models, and the pragmatic-explanatory continuum indicator summary tool for designing pragmatic trials. Discussion and critique of these health-related methodologies shows that these research methods are more suited for the philosophy and treatment options that naturopathy is based on.

CONCLUSIONS

Successful implementation of naturopathic research methodologies, and translation and dissemination of research will require a substantial paradigm shift in which naturopathic practitioners adopt a greater level of responsibility for developing an evidence base for naturopathic medicine.

Ensuring continuity of patient care across the healthcare interface: Telephone follow-up post-hospitalization

AIMS

To implement pharmacist-led, postdischarge telephone follow-up (TFU) intervention and to evaluate its impact on rehospitalization parameters in polypharmacy patients, via comparison with a well-matched control group.

METHOD

Pragmatic, prospective, quasi-experimental study. Intervention patients were matched by propensity score techniques with a control group. Guided by results from a pilot study, clinical pharmacists implemented TFU intervention, added to routine integrated medicines management service.

RESULTS

Using an intention to treat approach, reductions in 30- and 90-day readmission rates for intervention patients compared with controls were 9.9% [odds ratio = 0.57; 95% confidence interval (CI): 0.36-0.90; P < 0.001] and 15.2% (odds ratio = 0.53; 95% CI: 0.36-0.79; P = 0.021) respectively. Marginal mean time to readmission was 70.9 days (95% CI: 66.9-74.9) for intervention group compared with 60.1 days (95% CI: 55.4-64.7) for controls. Mean length of hospital stay compared with control was (8.3 days vs. 6.7 days; P < 0.001). Benefit: cost ratio for 30-day readmissions was 29.62, and 23.58 for 90-day interval. Per protocol analyses gave more marked improvements. In intervention patients, mean concern scale score, using Beliefs about Medicine Questionnaire, was reduced 3.2 (95% CI: -4.22 to -2.27; P < 0.001). Mean difference in Medication Adherence Report Scale was 1.4 (22.7 vs. 24.1; P < 0.001). Most patients (83.8%) reported having better control of their medicines after the intervention.

CONCLUSIONS

Pharmacist-led postdischarge structured TFU intervention can reduce 30- and 90-day readmission rates. Positive impacts were noted on time to readmission, length of hospital stay upon readmission, healthcare costs, patient beliefs about medicines, patient self-reported adherence and satisfaction.

Making Health Research Matter: A Call to Increase Attention to External Validity

Most of the clinical research conducted with the goal of improving health is not generalizable to nonresearch settings. In addition, scientists often fail to replicate each other's findings due, in part, to lack of attention to contextual factors accounting for their relative effectiveness or failure. To address these problems, we review the literature on assessment of external validity and summarize approaches to designing for generalizability. When investigators conduct systematic reviews, a critical need is often unmet: to evaluate the pragmatism and context of interventions, as well as their effectiveness. Researchers, editors, and grant reviewers can implement key changes in how they consider and report on external validity issues. For example, the recently published expanded CONSORT figure may aid scientists and potential program adopters in summarizing participation in and representativeness of a program across different settings, staff, and patients. Greater attention to external validity is needed to increase reporting transparency, improve program dissemination, and reduce failures to replicate research.

An Integrative, Systematic Review Exploring the Research, Effectiveness, Adoption, Implementation, and Maintenance of Interventions to Reduce Sedentary Behaviour in Office Workers

Sedentary behaviour is associated with poor health outcomes, and office-based workers are at significant health risk, as they accumulate large proportions of their overall sitting time at work. The aim of this integrated systematic review was to collate and synthesize published research on sedentary behaviour interventions in the workplace that have reported on at least one an aspect of the reach, effectiveness, adoption, implementation, and maintenance (RE-AIM) framework. Studies were included if they involved adult office workers, were conducted in an office setting, and changes in sedentary behaviour had been measured as a primary outcome. Five electronic databases were searched yielding 7234 articles, with 75 articles (61 individual interventions) meeting the inclusion criteria. Reach indicators were the most frequently reported RE-AIM dimensions, which were reported on average 59% of the time. Efficacy/effectiveness was the second most reported dimension at 49% reporting across all of the indicators. Implementation indicators were reported an average of 44% of the time, with indicators of adoption and maintenance reported as the lowest of all indicators at 13% and 8%, respectively. Recommendations are provided to improve reporting across all RE-AIM dimensions, which is an important first step to enable the effective translation of interventions into real world settings.

Association of Rapid Molecular Testing With Duration of Respiratory Isolation for Patients With Possible Tuberculosis in a US Hospital

IMPORTANCE

New guidelines recommend that molecular testing replace sputum-smear microscopy to guide discontinuation of respiratory isolation in patients undergoing evaluation for active tuberculosis (TB) in health care settings.

OBJECTIVE

To evaluate the implementation and impact of a molecular testing strategy to guide discontinuation of isolation.

DESIGN, SETTING, AND PARTICIPANTS

Prospective cohort study with a pragmatic, before-and-after-implementation design of 621 consecutive patients hospitalized at Zuckerberg San Francisco General Hospital and Trauma Center who were undergoing sputum examination for evaluation for active pulmonary TB from January 2014 to January 2016.

INTERVENTIONS

Implementation of a sputum molecular testing algorithm using GeneXpert MTB/RIF (Xpert; Cepheid) to guide discontinuation of isolation.

MAIN OUTCOMES AND MEASURES

We measured the proportion of patients with molecular testing ordered and completed; the accuracy of the molecular testing algorithm in reference to mycobacterial culture; the duration of each component of the testing and isolation processes; length of stay; mean days in isolation and in hospital; and mean cost. We extracted data from hospital records and compared measures before and after implementation.

RESULTS

Clinicians ordered sputum testing for TB for 621 patients at ZSFG during the 2-year study period. Of 301 patients in the preimplementation period with at least 1 sputum microscopy and culture ordered, clinicians completed the rapid TB testing evaluation process for 233 (77%).Among 320 patients evaluated in the postimplementation period, clinicians ordered molecular testing for 234 (73%) patients and received results for 295 of 302 (98%) tests ordered. Median age was 54 years (interquartile range, 44-63 years), and 161 (26%) were women. The molecular testing algorithm accurately diagnosed all 7 patients with culture-confirmed TB and excluded TB in all 251 patients with Mycobacterium tuberculosis (MTB) culture-negative results. Compared with the preimplementation period, there were significant decreases in median times to final rapid test result (39.1 vs 22.4 hours, P < .001), discontinuation of isolation (2.9 vs 2.5 days, P = .001), and hospital discharge (6.0 vs 4.9 days, P = .003), on average saving $13 347 per isolated TB-negative patient.

CONCLUSIONS AND RELEVANCE

A sputum molecular testing algorithm to guide discontinuation of respiratory isolation for patients undergoing evaluation for active TB was safe, feasible, widely and sustainably adopted, and provided substantial clinical and economic benefits. Molecular testing may facilitate more efficient, patient-centered evaluation for possible TB in US hospitals.

Using the RE-AIM framework to evaluate a school-based municipal programme tripling time spent on PE

Documenting the implementation of effective real-world programmes is considered an important step to support the translation of evidence into practice. Thus, the aim of this study was to identify factors influencing the adoption, implementation and maintenance of the Svendborgproject (SP) - an effective real-world programme comprising schools to implement triple the amount of physical education (PE) in pre-school to sixth grade in six primary schools in the municipality of Svendborg, Denmark. SP has been maintained for ten years and scaled up to all municipal schools since it was initiated in 2008. The Reach, Effectiveness, Adoption, Implementation and Maintenance framework (RE-AIM) was applied as an analytic tool through a convergent mixed method triangulation design. Results show that SP has been implemented with high fidelity and become an established part of the municipality and school identity. The successful implementation and dissemination of the programme has been enabled through the introduction of a predominantly bottom-up approach combined with simple non-negotiable requirements. The results show that this combination has led to a better fit of programmes to the individual school context while still obtaining high implementation fidelity. Finally, the early integration of research has legitimated and benefitted the programme.

Expanding the CONSORT Figure: Increasing Transparency in Reporting on External Validity

INTRODUCTION

There are major problems with failure to replicate research findings. Contributing to this problem is a failure to report on factors related to external validity. Frequently, researchers have little knowledge whether findings apply more generally, especially to low-resource settings and underserved populations. The CONSORT flow diagram has improved reporting on variables related to internal validity, but it has very limited detail on issues related to external validity. A recent CONSORT update and other publications have called for more transparent reporting on external validity and context, and information regarding the sustainability of interventions. All of these elements influence the generalizability of findings from outcomes research.

METHODS

Drawing on theory, a prior meeting, and recent recommendations for reporting factors related to external validity, the authors propose an expansion of the basic CONSORT flow diagram for clinical trials to concisely summarize these data that recent CONSORT statements and other guidelines have recommended.

RESULTS

The authors propose the use of an expanded CONSORT figure and illustrate its utility with an example. The expanded CONSORT figure adds data about participation and representativeness at the levels of settings and staff, and about intervention sustainability after project support ends. The authors provide an expanded CONSORT figure reporting template, and demonstrate its use.

CONCLUSIONS

Improving transparent reporting on external validity by using the proposed expanded CONSORT figure would help to address both the scientific replication crisis and health equity concerns. This figure provides a method to efficiently address the representativeness, generalizability, and sustainability of outcomes research.