Market viability: a neglected concept in implementation science

A systematic review of experimentally tested implementation strategies across health and human service settings: evidence from 2010-2022

BACKGROUND

Studies of implementation strategies range in rigor, design, and evaluated outcomes, presenting interpretation challenges for practitioners and researchers. This systematic review aimed to describe the body of research evidence testing implementation strategies across diverse settings and domains, using the Expert Recommendations for Implementing Change (ERIC) taxonomy to classify strategies and the Reach Effectiveness Adoption Implementation and Maintenance (RE-AIM) framework to classify outcomes.

METHODS

We conducted a systematic review of studies examining implementation strategies from 2010-2022 and registered with PROSPERO (CRD42021235592). We searched databases using terms "implementation strategy", "intervention", "bundle", "support", and their variants. We also solicited study recommendations from implementation science experts and mined existing systematic reviews. We included studies that quantitatively assessed the impact of at least one implementation strategy to improve health or health care using an outcome that could be mapped to the five evaluation dimensions of RE-AIM. Only studies meeting prespecified methodologic standards were included. We described the characteristics of studies and frequency of implementation strategy use across study arms. We also examined common strategy pairings and cooccurrence with significant outcomes.

FINDINGS

Our search resulted in 16,605 studies; 129 met inclusion criteria. Studies tested an average of 6.73 strategies (0-20 range). The most assessed outcomes were Effectiveness (n=82; 64%) and Implementation (n=73; 56%). The implementation strategies most frequently occurring in the experimental arm were Distribute Educational Materials (n=99), Conduct Educational Meetings (n=96), Audit and Provide Feedback (n=76), and External Facilitation (n=59). These strategies were often used in combination. Nineteen implementation strategies were frequently tested and associated with significantly improved outcomes. However, many strategies were not tested sufficiently to draw conclusions.

CONCLUSION

This review of 129 methodologically rigorous studies built upon prior implementation science data syntheses to identify implementation strategies that had been experimentally tested and summarized their impact on outcomes across diverse outcomes and clinical settings. We present recommendations for improving future similar efforts.

Implementation Requires Evaluation of Adoption: Lessons From a Multimodal Pain Regimen Order Set

INTRODUCTION

Multimodal pain regimen (MMPR) protocols are the standard of care per the 2020 Trauma Quality Improvement Program guidelines. MMPR implementation methodology in trauma services has not been reported. The primary objective of this study was to evaluate the adoption of an MMPR order set at a level 1 trauma center and to describe its implementation. We hypothesized that order set utilization would be about 50%, and barriers to adoption would be related to personal biases.

METHODS

This was a mixed-methods study at a level 1 trauma center. We retrospectively evaluated MMPR utilization from July 1, 2021 to February 28, 2022. Agile implementation was the method used to implement a clinical decision support tool for the MMPR: a flow chart order set in the electronic medical record. This methodology utilizes short experiment sprints during which data are collected to guide the next iterations. During this process quantitative as well as qualitative data were collected. This included end user testing of the order set and a survey distributed to surgical residents about the order set. Manual thematic network analysis was employed to identify basic and organizing themes from the survey responses.

RESULTS

A total of 587 trauma patients were admitted during the study period and 95 patients (16.2%) had MMPR ordered through the order set. The survey response rate was 19% (13/68). We identified ease of use, desire for options, inadequate education, and assumption of personal expertise as the four basic themes from the survey. These basic themes were further analyzed to two organizing themes: heuristics and overconfidence bias.

CONCLUSIONS

The MMPR order set was easy to use but had low adoption at our center in the first 8 months of implementation. Agile implementation methodology provided an ideal framework to identify reasons for low adoption and guide the next sprint to address personal biases, improve heuristics, and provide effective education and dissemination. Evaluation of utilization and qualitative analysis are key components to ensuring clinical decision support tool adoption.

Assessing the market viability of a packaged intensive health behavior and lifestyle treatment

In the USA, more than 14 million children are impacted by obesity. Despite intensive health behavior and lifestyle treatments being found effective, gaps exist in moving these interventions into widespread use. Focusing on market viability could improve the dissemination and sustainment of interventions. The purpose of this paper is to outline the process and results of our market viability assessment for the Healthy Weight Clinic (HWC), a Centers for Disease Control and Prevention-recognized Family Health Weight Program. We conducted a market viability assessment using the Speeding Research-test INTerventions (SPRINT) program to gain insights into the commercialization and marketplace for the HWC. Through the process of customer discovery, we interviewed 50 stakeholders to test our hypotheses pertaining to our business model. Key takeaways were the need for packaged interventions that offer support and training for providers, and interventions that are multidisciplinary and located within the medical home. We also learned that (i) the intervention goals must align with the healthcare organization's performance metrics; (ii) services need to be reimbursable; and (iii) the importance of understanding different customer segments (i.e. program users vs. organization decision-makers) and their unique needs. The market viability assessment is a critical step to transforming the HWC into a viable commercial product. The process we have outlined is replicable by others and by encouraging other teams to design for dissemination we can increase the number of evidence-based, packaged IHBLTs available to children with obesity.

From formative design to service-ready therapeutic: A pragmatic approach to designing digital mental health interventions across domains

As digital mental health interventions (DMHIs) proliferate, there is a growing need to understand the complexities of moving these tools from concept and design to service-ready products. We highlight five case studies from a center that specializes in the design and evaluation of digital mental health interventions to illustrate pragmatic approaches to the development of digital mental health interventions, and to make transparent some of the key decision points researchers encounter along the design-to-product pipeline. Case studies cover different key points in the design process and focus on partnership building, understanding the problem or opportunity, prototyping the product or service, and testing the product or service. We illustrate lessons learned and offer a series of questions researchers can use to navigate key decision points in the digital mental health intervention (DMHI) development process.

Implementation and dissemination of home- and community-based interventions for informal caregivers of people living with dementia: a systematic scoping review

BACKGROUND

Informal caregivers of people with dementia (PwD) living at home are often the primary source of care, and, in their role, they often experience loss of quality of life. Implementation science knowledge is needed to optimize the real-world outcomes of evidence-based interventions (EBIs) for informal caregivers. This scoping review aims to systematically synthesize the literature that reports implementation strategies employed to deliver home- and community-based EBIs for informal caregivers of PwD, implementation outcomes, and the barriers and facilitators to implementation of these EBIs.

METHODS

Embase, MEDLINE, Web of Science, and Cochrane Library were searched from inception to March 2021; included studies focused on "implementation science," "home- and community-based interventions," and "informal caregivers of people with dementia." Titles and abstracts were screened using ASReview (an innovative AI-based tool for evidence reviews), and data extraction was guided by the ERIC taxonomy, the Implementation Outcome Framework, and the Consolidated Framework for Implementation Science Research; each framework was used to examine a unique element of implementation.

RESULTS

Sixty-seven studies were included in the review. Multicomponent (26.9%) and eHealth (22.3%) interventions were most commonly reported, and 31.3% of included studies were guided by an implementation science framework. Training and education-related strategies and provision of interactive assistance were the implementation strategy clusters of the ERIC taxonomy where most implementation strategies were reported across the reviewed studies. Acceptability (82.1%), penetration (77.6%), and appropriateness (73.1%) were the most frequently reported implementation outcomes. Design quality and packaging (intervention component suitability) and cosmopolitanism (partnerships) constructs, and patient's needs and resources and available resources (infrastructure) constructs as per the CFIR framework, reflected the most frequently reported barriers and facilitators to implementation.

CONCLUSION

Included studies focused largely on intervention outcomes rather than implementation outcomes and lacked detailed insights on inner and outer setting determinants of implementation success or failure. Recent publications suggest implementation science in dementia research is developing but remains in nascent stages, requiring future studies to apply implementation science knowledge to obtain more contextually relevant findings and to structurally examine the mechanisms through which implementation partners can strategically leverage existing resources and regional networks to streamline local implementation. Mapping local evidence ecosystems will facilitate structured implementation planning and support implementation-focused theory building.

TRIAL REGISTRATION

Not applicable.

The role of implementation organizations in scaling evidence-based psychosocial interventions

BACKGROUND

To bring evidence-based interventions (EBIs) to individuals with behavioral health needs, psychosocial interventions must be delivered at scale. Despite an increasing effort to implement effective treatments in communities, most individuals with mental health and behavioral problems do not receive EBIs. We posit that organizations that commercialize EBIs play an important role in disseminating EBIs, particularly in the USA. The behavioral health and implementation industry is growing, bringing the implementation field to an important inflection point: how to scale interventions to improve access while maintaining EBI effectiveness and minimizing inequities in access to psychosocial intervention.

MAIN BODY

We offer a first-hand examination of five illustrative organizations specializing in EBI implementation: Beck Institute for Cognitive Behavioral Therapy; Incredible Years, Inc.; the PAXIS Institute; PracticeWise, LLC; and Triple P International. We use the Five Stages of Small Business Growth framework to organize themes. We discuss practical structures (e.g., corporate structures, intellectual property agreements, and business models) and considerations that arise when trying to scale EBIs including balancing fidelity and reach of the intervention. Business models consider who will pay for EBI implementation and allow organizations to scale EBIs.

CONCLUSION

We propose research questions to guide scaling: understanding the level of fidelity needed to maintain efficacy, optimizing training outcomes, and researching business models to enable organizations to scale EBIs.

Financial Impact of a Culturally Sensitive Hispanic Kidney Transplant Program on Increasing Living Donation

BACKGROUND

In the United States, Hispanic/Latinx patients receive disproportionately fewer living donor kidney transplants (LDKTs) than non-Hispanic White patients. Northwestern Medicine's culturally targeted Hispanic Kidney Transplant Program (HKTP) was found to increase LDKTs in Hispanic patients at 1 of 2 transplant programs with greater implementation fidelity.

METHODS

We conducted a budget impact analysis to evaluate HKTP's impact on program financial profiles from changes in volume of LDKTs and deceased donor kidney transplants (DDKTs) in 2017 to 2019. We estimated HKTP programmatic costs, and kidney transplant (KT) program costs and revenues. We forecasted transplant volumes, HKTP programmatic costs, and KT program costs and revenues for 2022-2024.

RESULTS

At both programs, HKTP programmatic costs had <1% impact on total KT program costs, and HKTP programmatic costs comprised <1% of total KT program revenues in 2017-2019. In particular, the total volume of Hispanic KTs and HKTP LDKTs increased at both sites. Annual KT program revenues of HKTP LDKTs and DDKTs increased by 226.9% at site A and by 1042.9% at site B when comparing 2019-2017. Forecasted HKTP LDKT volume showed an increase of 36.4% (site A) and 33.3% (site B) with a subsequent increase in KT program revenues of 42.3% (site A) and 44.3% (site B) among HKTP LDKTs and DDKTs.

CONCLUSIONS

HKTP programmatic costs and KT evaluation costs are potentially recoverable by reimbursement of organ acquisition costs and offset by increases in total KT program revenues of LDKTs; transplant programs may find implementation of the HKTP financially manageable.

FAST: A Framework to Assess Speed of Translation of Health Innovations to Practice and Policy

The 17-year time span between discovery and application of evidence in practice has become a unifying challenge for implementation science and translational science more broadly. Further, global pandemics and social crises demand timely implementation of rapidly accruing evidence to reduce morbidity and mortality. Yet speed remains an understudied metric in implementation science. Prevailing evaluations of implementation lack a temporal aspect, and current approaches have not yielded rapid implementation. In this paper, we address speed as an important conceptual and methodological gap in implementation science. We aim to untangle the complexities of studying implementation speed, offer a framework to assess speed of translation (FAST), and provide guidance to measure speed in evaluating implementation. To facilitate specification and reporting on metrics of speed, we encourage consideration of stakeholder perspectives (e.g., comparison of varying priorities), referents (e.g., speed in attaining outcomes, transitioning between implementation phases), and observation windows (e.g., time from intervention development to first patient treated) in its measurement. The FAST framework identifies factors that may influence speed of implementation and potential effects of implementation speed. We propose a research agenda to advance understanding of the pace of implementation, including identifying accelerators and inhibitors to speed.

It's all in the name: why exposure therapy could benefit from a new one

Exposure therapy for anxiety and related disorders is the psychological intervention with the strongest support for its efficacy and effectiveness to date. Yet, it is the least used evidence-based intervention in routine clinical practice, with a long-acknowledged public relations problem. Despite a wealth of research aimed at improving uptake of exposure, exposure's marketing and branding remains an untapped target. We first introduce principles from the marketing literature to propose that the field take steps toward a rebranding and repackaging of exposure therapy to support efforts to implement it widely. Second, we present preliminary data on clinician preferences for the use of alternative terminology developed to be more palatable and marketable - "Supported Approach of Feared Experiences - Cognitive Behavioral Therapy (SAFE-CBT)" - compared to traditional terminology. This initial survey indicated that most clinicians preferred use of the SAFE-CBT term when talking to patients, whereas only a minority preferred it for use among training clinicians. We conclude by discussing implications of these results for future efforts to implement exposure therapy more widely and set an agenda for future research in this space.

Promoting rigor and sustainment in implementation science capacity building programs: A multi-method study

Background

The field of Implementation science (IS) continues to evolve, and the number and type of IS capacity building Programs (ISCBPs) are in flux. These changes push the field to revisit the accepted IS competencies and to guide sustainment of ISCBPs. Our objectives were: (1) compare characteristics of current ISCBPs; (2) identify recommendations to support ISCBP sustainment; (3) measure how often ISCBPs address IS competencies; (4) identify novel and important IS competencies for the field.

Method

This multi-method study included ISCBPs delivering structured, longitudinal IS training, excluding single courses and brief workshops. We used three complementary methods to meet our objectives. First, we identified ISCBPs via an internet search and snowball sampling methods. Second, we surveyed these ISCBPs to identify areas of program focus, types of trainees, IS competencies addressed, and recommendations to sustain ISCBPs. Third, we conducted a modified Delphi process with IS researchers/leaders to reach consensus on the IS competencies that were both important and novel as compared to the IS competencies published to date.

Results

Among 74 eligible ISCBPs identified, 46 responded (62% response rate). Respondent ISCBPs represented diverse areas of focus (e.g., global health, cardiopulmonary disease) and trainee stages (e.g., graduate students, mid-career faculty). While most respondent ISCBPs addressed core IS methods, targeting IS competencies was less consistent (33% for nongraduate/non-fellowship ISCBPs; >90% for graduate/national ISCBPs). Our modified Delphi process identified eight novel and important IS competencies related to increasing health equity or the speed of translation. Recommendations to sustain ISCBPs included securing financial administrative support.

Conclusions

Current ISCBPs train learners across varying career stages in diverse focus areas. To promote rigor, we recommend ISCBPs address specific IS competencies, with consideration of these eight novel/emerging competencies. We also recommend ISCBPs report on their IS competencies, focus area(s), and trainee characteristics. ISCBP programs need administrative financial support.

Plain Language Summary

There is a limited workforce capacity to conduct implementation science (IS) research. To address this gap, the number and type of IS capacity building Programs (ISCBPs) focusing on training researchers and practitioners in IS methods continue to increase. Our efforts to comprehensively identify and describe ISCBPs for researchers and practitioners highlighted four implications for leaders of ISCBPs related to program sustainment and rigor. First, we identified a range of contextual characteristics of ISCBPs, including the research topics, methods, and IS competencies addressed, and the types of trainees accepted. Second, given the variability of trainee types and research, rigorous ISCBP programs should tailor the IS competencies and methods addressed to the skills needed by the types of trainees in their program. Third, the field of IS needs to periodically revisit the competencies needed with attention to the skills needed in the field. We used a consensus-building process with ISCBP leaders and other IS experts to expand existing IS competencies and identified eight important, novel IS competencies that broadly relate to promoting health equity and speeding the translation of research to practice. Finally, as more institutions consider developing ISCBPs, we identified factors needed to support ISCBP sustainment, including ongoing financial support. In addition to these implications for ISCBP leaders, there are also policy implications. For example, IS journals may enact policies to require manuscripts evaluating ISCBP performance to report on certain contextual characteristics, such as the IS competencies addressed and types of trainees accepted. The field may also consider developing an accreditation body to evaluate the rigor of ISCBP curricula.