Multi-method process maps: An interdisciplinary approach to investigate ad hoc modifications in protocol-driven interventions

Optimization of implementation strategies using the Multiphase Optimization STratgey (MOST) framework: Practical guidance using the factorial design

The Multiphase Optimization STrategy (MOST) is a framework that uses three phases-preparation, optimization, and evaluation-to develop multicomponent interventions that achieve intervention EASE by strategically balancing Effectiveness, Affordability, Scalability, and Efficiency. In implementation science, optimization of the intervention requires focus on the implementation strategies-things that we do to deliver the intervention-and implementation outcomes. MOST has been primarily used to optimize the components of the intervention related to behavioral or health outcomes. However, innovative opportunities to optimize discrete (i.e. single strategy) and multifaceted (i.e. multiple strategies) implementation strategies exist and can be done independently, or in conjunction with, intervention optimization. This article details four scenarios where the MOST framework and the factorial design can be used in the optimization of implementation strategies: (i) the development of new multifaceted implementation strategies; (ii) evaluating interactions between program components and a discrete or multifaceted implementation strategies; (iii) evaluating the independent effects of several discrete strategies that have been previously evaluated as a multifaceted implementation strategy; and (iv) modification of a discrete or multifaceted implementation strategy for the local context. We supply hypothetical school-based physical activity examples to illustrate these four scenarios, and we provide hypothetical data that can help readers make informed decisions derived from their trial data. This manuscript offers a blueprint for implementation scientists such that not only is the field using MOST to optimize the effectiveness of an intervention on a behavioral or health outcome, but also that the implementation of that intervention is optimized.

Identifying opportunities to optimize mass drug administration for soil-transmitted helminths: A visualization and descriptive analysis using process mapping

BACKGROUND

The control of soil-transmitted helminths (STH) is achieved through mass drug administration (MDA) with deworming medications targeting children and other high-risk groups. Recent evidence suggests that it may be possible to interrupt STH transmission by deworming individuals of all ages via community-wide MDA (cMDA). However, a change in delivery platforms will require altering implementation processes.

METHODS

We used process mapping, an operational research methodology, to describe the activities required for effective implementation of school-based and cMDA in 18 heterogenous areas and over three years in Benin, India, and Malawi. Planned activities were identified during workshops prior to initiation of a large cMDA trial (the DeWorm3 trial). The process maps were updated annually post-implementation, including adding or removing activities (e.g., adaptations) and determining whether activities occurred according to plan. Descriptive analyses were performed to quantify differences and similarities at baseline and over three implementation years. Comparative analyses were also conducted between study sites and areas implementing school-based vs. cMDA. Digitized process maps were developed to provide a visualization of MDA processes and inspected to identify implementation bottlenecks and inefficient activity flows.

RESULTS

Across three years and all clusters, implementation of cMDA required an average of 13 additional distinct activities and was adapted more often (5.2 adaptations per year) than school-based MDA. An average of 41% of activities across both MDA platforms did not occur according to planned timelines; however, deviations were often purposeful to improve implementation efficiency or effectiveness. Visualized process maps demonstrated that receipt of drugs at the local level may be an implementation bottleneck. Many activities rely on the effective setting of MDA dates and estimating quantity of drugs, suggesting that the timing of these activities is important to meet planned programmatic outcomes.

CONCLUSION

Implementation processes were heterogenous across settings, suggesting that MDA is highly context and resource dependent and that there are many viable ways to implement MDA. Process mapping could be deployed to support a transition from a school-based control program to community-wide STH transmission interruption program and potentially to enable integration with other community-based campaigns.

TRIAL REGISTRATION

NCT03014167.

Reproducibility of resistance exercise therapy interventions in trials for chronic low back pain is challenging: A systematic review

OBJECTIVE

Determine reproducibility of resistance exercise regimens in trials for CLBP and determine if recently available checklists are effective.

METHODS

Four databases (Medline, PubMed, Cochrane and CINAHL) were searched for keywords related to back pain and resistance exercise. Reproducibility was assessed using two checklists, the 12-item Template for Intervention Description and Replication (TIDieR) and the 19-item Consensus on Exercise Reporting Template (CERT). The proportion reporting was analysed, with additional comparison of trials pre- and post-availability of each checklist. A generalised linear regression was conducted with checklist items as the dependent variable and year of publication as the independent (PROSPERO ID = #CRD42020186036).

RESULTS

Overall, details that facilitate reproducibility were under-reported. No trials reported all checklist items, while only 18 trials (35.5%) and 5 trials (9.8%) reported 75%+ of checklist items for the TIDieR and CERT, respectively. A median of 8 (IQR 2) of 12 TIDieR criteria were reported and a median of 9 (IQR 7) of 19 criteria were reported for the CERT. There was no difference pre/post checklist publication (TIDieR median before = 8 (IQR 2), after = 8 (IQR 2.25); CERT mean before = 9 (IQR 5.25), after = 9 (IQR 7)). Regression failed to support improved reporting over time. The majority of studies (86.3%) were scored as having an elevated risk of bias.

CONCLUSIONS

Reproducibility of resistance exercise in CLBP trials appears questionable due to low levels of reporting. The publication reporting checklists have not resulted in improvement. Real-world reproducibility is questionable. There is a need to improve reporting to maximise reproducibility.

IMPACT STATEMENT

The present results reveal a demand in improved reporting to ensure both enhanced clinical translation in the real-world and replicability to enhance knowledge of best-practice for resistance exercise in the CLBP population.

The quick pivot: Capturing real world modifications for the re-implementation of an early psychosis program transitioning to virtual delivery

Background

Team-based Early Psychosis Intervention (EPI) services is standard of care for youth with psychosis. The COVID-19 pandemic required most EPI services to mount an unplanned, rapid pivot to virtual delivery, with limited guidance on how to deliver virtual clinical services or whether quality of re-implementation and treatment outcomes would be impacted. We used a structured approach to identify essential modifications for the delivery of core components and explored facilitators and barriers for re-implementation and fidelity of a virtually delivered EPI intervention.

Materials and methods

NAVIGATE is a structured approach to team-based EPI. It provides detailed modules to guide delivery of core components including medication management, psychoeducation and psychotherapies, supported employment/education, and family education. Having initially implemented NAVIGATE at the Centre for Addiction and Mental Health (CAMH) in 2017, the EPI service transitioned to virtual delivery amid the COVID pandemic. Using a practice profile developed to support implementation, we detailed how core components of NAVIGATE were rapidly modified for virtual delivery as reported in structured group meetings with clinicians. The Framework for Reporting Adaptations and Modifications for Evidence-Based Interventions (FRAME) was used to describe modifications. Fidelity to the EPI standards of care was assessed by the First Episode Psychosis Fidelity Scale (FEPS-FS). Re-implementation barriers and facilitators and subsequent mitigation strategies were explored using structured clinician interviews guided by the Consolidated Framework for Implementation Research (CFIR).

Results

Identified modifications related to the intervention process, context, and training. We identified contextual factors affecting the re-implementation of virtually delivered NAVIGATE and then documented mitigating strategies that addressed these barriers. Findings can inform the implementation of virtual EPI services elsewhere, including guidance on processes, training and technology, and approaches to providing care virtually.

Discussion

This study identified modifications, impacts and mitigations to barriers emerging from rapid, unplanned virtual delivery of EPI services. These findings can support delivery of high-quality virtual services to youth with psychosis when virtual care is indicated.

Using a longitudinal multi-method approach to document, assess, and understand adaptations in the Veterans Health Administration Advanced Care Coordination program

Background

Understanding adaptations supports iterative refinement of the implementation process and informs scale out of programs. Systematic documentation of adaptations across the life course of programs is not routinely done, and efficient capture of adaptations in real world studies is not well understood.

Methods

We used a multi-method longitudinal approach to systematically document adaptations during pre-implementation, implementation, and sustainment for the Veteran Health Administration (VA) Advanced Care Coordination program. This approach included documenting adaptations through a real-time tracking instrument, process maps, Implementation and Evaluation (I&E) team meeting minutes, and adaptation interviews. Data collection was guided by the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) enhanced framework for reporting adaptations and modifications to evidence-based interventions (FRAME) model. Adaptations were evaluated across 9 categories, and analytic team consensus and member-checking were used to validate the results.

Results

A total of 144 individual adaptations were identified across two implementation sites and the four data sources; analytic team consensus and member-checking processes resulted in 50 unique adaptations. Most adaptations took place during the early implementation and mid-implementation phases and were: 1) planned; 2) made to address changes in program delivery; 3) made to extend a component; 4) related to the core component of the intervention concerning notification of the community emergency department visit; 5) initiated by the entire or most of the I&E team; 6) made on the basis of: pragmatic/practical considerations; 7) made with an intent to improve implementation domain (to make the intervention delivered more consistently; to better fit the local practice, patient flow or Electronic Health Record (EHR) and/or for practical reasons); 8) a result of internal influences; 9) perceived to impact the RE-AIM implementation dimension (consistent delivery of quality care or costs). I&E team meeting minutes and process maps captured the highest numbers of unique adaptations (n = 19 and n = 13, respectively).

Conclusion

Our longitudinal, multi-method approach provided a feasible way to collect adaptations data through engagement of multiple I&E team members, allowing and a broader understanding of adaptations that took place. Recommendations for future research include pragmatic assessment of the impact of adaptations and meaningful data collection without overburdening the implementing teams and front-line staff.

Implementation Science Research in Communication Sciences and Disorders: A Scoping Review

PURPOSE

The purpose of this study was to complete a scoping review of implementation science (IS) research in communication sciences and disorders (CSD) over time and to determine characteristics of IS research in CSD.

METHOD

A scoping review was conducted of PubMed and Education Resources Information Center for sources published in English that (a) included CSD practitioners, (b) addressed IS research, and (c) identified a specific evidence-based practice. Resulting sources were systematically examined for study aim, patient populations, implementation framework utilized, setting of the study, implementation strategy examined, and implementation outcome measured.

RESULTS

The majority of the 82 studies that underwent a full-text review (80.5%) were published in 2014 or later. One fourth of the studies were concept papers, and another one fourth focused on context assessment (25.6% of studies, each), 11% focused on designing implementation strategies, and 36.6% focused on testing implementation strategies. The patient population most frequently represented aphasia (21.3%), and most studies (34.4%) were conducted in inpatient medical settings. Nearly half (42.6%) of the nonconcept studies lacked an IS framework. Among implementation strategies identified, approximately one third of studies focused on education and/or training plus another strategy and one fourth focused on education and/or training alone. Implementation outcomes measured typically represented early stages of implementation.

CONCLUSIONS

This scoping review of IS research in CSD described the landscape of IS studies in CSD. IS is intersecting with CSD at a rapid rate, especially since 2014. Future IS research in CSD should adopt an implementation framework a priori and consider the broad range of implementation strategies and outcomes to support the uptake of research into typical practice settings.

Integrating Pediatric Universal Behavioral Health Care at Federally Qualified Health Centers

BACKGROUND

Research supports integrated pediatric behavioral health (BH), but evidence gaps remain in ensuring equitable care for children of all ages. In response, an interdisciplinary team codeveloped a stepped care model that expands BH services at 3 federally qualified health centers (FQHCs).

METHODS

FQHCs reported monthly electronic medical record data regarding detection of BH issues, receipt of services, and psychotropic medications. Study staff reviewed charts of children with attention-deficit/hyperactivity disorder (ADHD) before and after implementation.

RESULTS

Across 47 437 well-child visits, >80% included a complete BH screen, significantly higher than the state's long-term average (67.5%; P < .001). Primary care providers identified >30% of children as having BH issues. Of these, 11.2% of children <5 years, 53.8% of 5-12 years, and 74.6% >12 years were referred for care. Children seen by BH staff on the day of referral (ie, "warm hand-off") were more likely to complete an additional BH visit than children seen later (hazard ratio = 1.37; P < .0001). There was no change in the proportion of children prescribed psychotropic medications, but polypharmacy declined (from 9.5% to 5.7%; P < .001). After implementation, diagnostic rates for ADHD more than doubled compared with baseline, follow-up with a clinician within 30 days of diagnosis increased (62.9% before vs 78.3% after; P = .03) and prescriptions for psychotropic medication decreased (61.4% before vs 43.9% after; P = .03).

CONCLUSIONS

Adding to a growing literature, results demonstrate that integrated BH care can improve services for children of all ages in FQHCs that predominantly serve marginalized populations.

Effectiveness of Screening in Early Intervention Settings to Improve Diagnosis of Autism and Reduce Health Disparities

IMPORTANCE

The American Academy of Pediatrics recommends referring children at elevated risk of autism spectrum disorder (ASD) for Part C early intervention (EI) services, but notes that EI services often fail to provide ASD screening.

OBJECTIVE

To evaluate the hypothesis that a multistage screening protocol for ASD implemented in 3 EI settings will increase autism detection, especially among Spanish-speaking families.

DESIGN, SETTING, AND PARTICIPANTS

Difference-in-differences analyses with propensity score weighting of a quasi-experimental design using administrative data on 3 implementation EI agencies and 9 comparison EI agencies from 2012 to 2018 provided by the Massachusetts Department of Public Health. Eligible children were aged 14 to 36 months, enrolled in EI, had no prior ASD diagnosis or medical condition precluding participation, and had parents who spoke English or Spanish. The final analytic sample included 33 326 unique patients assessed across 150 200 person-quarters.

EXPOSURES

Multistage ASD assessment protocol including ASD screening questionnaires, observational screener, and diagnostic evaluation.

MAIN OUTCOMES AND MEASURES

Increased incidence of ASD diagnoses as documented in Department of Public Health records and reductions in language-associated health care disparities.

RESULTS

Implementation of screening at 3 EI sites was associated with a significant increase in the rate of ASD diagnoses (incidence rate ratios [IRR], 1.6; 95% CI, 1.3-2.1; P < .001), representing an additional 8.1 diagnoses per 1000 children per quarter. Among Spanish-speaking families, screening was also associated with a significant increase in the rate of ASD diagnoses (IRR, 2.6; 95% CI, 1.6-4.3; P < .001), representing 15.4 additional diagnoses per 1000 children per quarter-a larger increase than for non-Spanish-speaking families (interaction IRR, 1.8; 95% CI, 1.0-3.1; P = .005). Exploratory analyses revealed that screening was associated with a larger increase in the rate of ASD diagnoses among boys (IRR, 1.8; 95% CI, 1.4-2.3; P < .001) than among girls (IRR, 1.1; 95% CI, 0.6-1.7; P = .84).

CONCLUSIONS AND RELEVANCE

In this study, associations between increased rates of ASD diagnoses and reductions in disparities for Spanish-speaking households support the effectiveness of multistage screening in EI. This study provides a comprehensive evaluation of ASD screening in EI settings as well as a rigorous evaluation of ASD screening in any setting with a no-screening comparison condition. Given that the intervention included multiple components, mechanisms of action warrant further research, as do disparities by child sex.

A National Survey of Patient Completion of Cognitive Behavioral Therapy for Chronic Pain: The Role of Therapist Characteristics, Attempt Rates, and Modification

The implementation of evidence-based psychotherapies, including patient-level measures such as penetration and rates of successfully completing a course of therapy, has received increasing attention. While much attention has been paid to the effect of patient-level factors on implementation, relatively little attention has been paid to therapist factors (e.g., professional training, experience).

Objective

The current study explores therapists' decisions to offer a particular evidence-based psychotherapy (cognitive behavioral therapy for chronic pain; CBT-CP), whether and how they modify CBT-CP, and the relationship between these decisions and patient completion rates.

Methods

The study utilized survey responses from 141 Veterans Affairs therapists certified in CBT-CP.

Results

Therapists reported attempting CBT-CP with a little less than one half of their patients with chronic pain (mean = 48.8%, s.d.=35.7). Therapist were generally split between reporting modifying CBT-CP for either very few or most of their patients. After controlling for therapist characteristics and modification, therapist-reported percentage of patients with attempted CBT-CP was positively associated with completion rates, t (111) = 4.57, p<.001.

Conclusions

Therapists who attempt CBT-CP more frequently may experience better completion rates, perhaps due to practice effects or contextual factors that support both attempts and completion. Future research should examine this relationship using objective measures of attempt rates and completion.

Extent, Type and Reasons for Adaptation and Modification When Scaling-Up an Effective Physical Activity Program: Physical Activity 4 Everyone (PA4E1)

Background: Few studies have described the extent, type and reasons for making changes to a program prior to and during its delivery using a consistent taxonomy. Physical Activity 4 Everyone (PA4E1) is a secondary school physical activity program that was scaled-up for delivery to a greater number of schools. We aimed to describe the extent, type and reasons for changes to the PA4E1 program (the evidence-based physical activity practices, implementation support strategies and evaluation methods) made before its delivery at scale (adaptations) and during its delivery in a scale-up trial (modifications). Methods: The Framework for Reporting Adaptations and Modifications-Enhanced (FRAME) was used to describe adaptations (planned and made prior to the scale-up trial) and modifications (made during the conduct of the trial). A list of adaptations was generated from a comparison of the efficacy and scale-up trials via published PA4E1 protocols, trial registrations and information provided by trial investigators. Monthly trial team meetings tracked and coded modifications in "real-time" during the conduct of the scale-up trial. The extent, type and reasons for both adaptations and modifications were summarized descriptively. Results: In total, 20 adaptations and 20 modifications were identified, these were to physical activity practices (n = 8; n = 3), implementation support strategies (n = 6; n = 16) and evaluation methods (n = 6, n = 1), respectively. Few adaptations were "fidelity inconsistent" (n = 2), made "unsystematically" (n = 1) and proposed to have a "negative" impact on the effectiveness of the program (n = 1). Reasons for the adaptations varied. Of the 20 modifications, all were "fidelity consistent" and the majority were made "proactively" (n = 12), though most were "unsystematic" (n = 18). Fifteen of the modifications were thought to have a "positive" impact on program effectiveness. The main reason for modification was the "available resources" (n = 14) of the PA4E1 Implementation Team. Conclusions: Adaptations and modifications to public health programs are common. Modifications have the potential to impact the implementation and effectiveness of programs. Our findings underscore the importance of comprehensive reporting of the extent, type and reasons for modifications as part of process evaluations, as this data may be important to the interpretation of trial findings. Clinical Trial Registration: https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=372870, Identifier ACTRN12617000681358.

Understanding How Parents Make Meaning of Their Child's Behaviors During Screening for Autism Spectrum Disorders: A Longitudinal Qualitative Investigation

A family's journey in understanding their child's behaviors in relation to Autism Spectrum Disorders (ASD) frequently begins with screening. This study aimed to characterize the interpretive processes that unfold for parents. We employed longitudinal interviews with 19 families engaged in a community-based multi-stage screening protocol. Parents participated in 1-6 interviews dependent upon children's length of engagement in the screening protocol; data were analyzed through modified grounded theory. Parents who moved towards understanding their child's behaviors as ASD expressed (1) sensitization to ASD symptoms, (2) differentiation from other developmental conditions, and (3) use of the ASD diagnosis to explain the etiology of concerning behaviors. Identifying interpretive processes involved during ASD screening provides new opportunities for shared decision-making.