Applying the lessons of implementation science to maximize feasibility and usability in team science intervention development

Using a Community-Informed Translational Model to Prioritize Translational Benefits in Youth Concussion Return-to-Learn Programs

BACKGROUND

The Translational Science Benefit Model (TSBM) was developed to broadly capture systematic measures of health and societal benefits from scientific research, beyond traditional outcome measures. We aimed to develop a systematic process for the application of the TSBM and to then provide an example of a novel application of the TSBM to an ongoing Return-to-Learn (RTL) after youth concussion project involving partnerships with community stakeholders.

METHODS

We invited investigators, project advisory board, and participants of the RTL project to participate in a modified Delphi process. We first generated a list of potential translational benefits using the indicators of the TSBM as guideposts. We then prioritized the benefits on an adapted Eisenhower matrix.

RESULTS

We invited 35 concussion care or research experts to participate, yielding 20 ranked translational benefits. Six of these recommendations were ranked high priority, six were regarded as investments, and eight were ranked as either low yield or low priority.

DISCUSSION

This study found that activities such as education and training of stakeholders, development of policy and consensus statements, and innovation in dissemination, were perceived as higher priority than other activities. Our approach using a modified Delphi process and incorporating the TSBM can be replicated to generate and prioritize potential benefits to society from research studies.

Use of implementation mapping in the planning of a hybrid type 1 pragmatic clinical trial: the BeatPain Utah study

BACKGROUND

Considerable disparities in chronic pain management have been identified. Persons in rural, lower income, and minoritized communities are less likely to receive evidence-based, nonpharmacologic care. Telehealth delivery of nonpharmacologic, evidence-based interventions for persons with chronic pain is a promising strategy to lessen disparities, but implementation comes with many challenges. The BeatPain Utah study is a hybrid type 1 effectiveness-implementation pragmatic clinical trial investigating telehealth strategies to provide nonpharmacologic care from physical therapists to persons with chronic back pain receiving care in ommunity health centers (CHCs). CHCs provide primary care to all persons regardless of ability to pay. This paper outlines the use of implementation mapping to develop a multifaceted implementation plan for the BeatPain study.

METHODS

During a planning year for the BeatPain trial, we developed a comprehensive logic model including the five-step implementation mapping process informed by additional frameworks and theories. The five iterative implementation mapping steps were addressed in the planning year: (1) conduct needs assessments for involved groups; (2) identify implementation outcomes, performance objectives, and determinants; (3) select implementation strategies; (4) produce implementation protocols and materials; and (5) evaluate implementation outcomes.

RESULTS

CHC leadership/providers, patients, and physical therapists were identified as involved groups. Barriers and assets were identified across groups which informed identification of performance objectives necessary to implement two key processes: (1) electronic referral of patients with back pain in CHC clinics to the BeatPain team and (2) connecting patients with physical therapists providing telehealth. Determinants of the performance objectives for each group informed our choice of implementation strategies which focused on training, education, clinician support, and tailoring physical therapy interventions for telehealth delivery and cultural competency. We selected implementation outcomes for the BeatPain trial to evaluate the success of our implementation strategies.

CONCLUSIONS

Implementation mapping provided a comprehensive and systematic approach to develop an implementation plan during the planning phase for our ongoing hybrid effectiveness-implementation trial. We will be able to evaluate the implementation strategies used in the BeatPain Utah study to inform future efforts to implement telehealth delivery of evidence-based pain care in CHCs and other settings.

TRIAL REGISTRATION

ClinicalTrials.gov Identifier: NCT04923334 . Registered June 11, 2021.

Use of implementation mapping in the planning of a hybrid type 1 pragmatic clinical trial: the BeatPain Utah study

Background

Considerable disparities in chronic pain management have been identified. Persons in rural, lower income and minoritized communities are less likely to receive evidence-based, nonpharmacologic care. Telehealth delivery of nonpharmacologic, evidence-based interventions for persons with chronic pain is a promising strategy to lessen disparities, but implementation comes with many challenges. The BeatPain Utah study is a hybrid type I effectiveness-implementation pragmatic clinical trial investigating telehealth strategies to provide nonpharmacologic care from physical therapists to persons with chronic back pain receiving care in Community Health Centers (CHCs). CHCs provide primary care to all persons regardless of ability to pay. This paper outlines the use of implementation mapping to develop a multifaceted implementation plan for the BeatPain study.

Methods

During a planning year for the BeatPain trial we developed a comprehensive logic model including the 5-step implementation mapping process informed by additional frameworks and theories. The five iterative implementation mapping steps were addressed in the planning year; 1) conduct needs assessments for involved groups; 2) identify implementation outcomes, performance objectives and determinants; 3) select implementation strategies; 4) produce implementation protocols and materials; and 5) evaluate implementation outcomes.

Results

CHC leadership/providers, patients and physical therapists were identified as involved groups. Barriers and assets were identified across groups which informed identification of performance objectives necessary to implement two key processes; 1) electronic referral of patients with back pain in CHC clinics to the BeatPain team; and 2) connecting patients with physical therapists providing telehealth. Determinants of the performance objectives for each group informed our choice of implementation strategies which focused on training, education, clinician support and tailoring physical therapy interventions for telehealth delivery and cultural competency. We selected implementation outcomes for the BeatPain trial to evaluate the success of our implementation strategies.

Conclusions

Implementation mapping provided a comprehensive and systematic approach to develop an implementation plan during the planning phase for our ongoing hybrid effectiveness-implementation trial. We will be able to evaluate the implementation strategies used in the BeatPain Utah study to inform future efforts to implement telehealth delivery of evidence-based pain care in CHCs and other settings.

Trial registration

Clinicaltrials.gov Identifier: NCT04923334. Registered June 11, 2021 (https://clinicaltrials.gov/study/NCT04923334.

Focus Group Study of Medical Stakeholders to Inform the Development of Resilient Together for Dementia: Protocol for a Postdiagnosis Live Video Dyadic Resiliency Intervention

BACKGROUND

Alzheimer disease and related dementias (ADRD) are increasingly common conditions that disrupt the lives of persons living with dementia and their spousal care partners. At the time of ADRD diagnoses, many couples experience challenges that produce emotional distress and relationship strain. At present, there are no interventions to address these challenges early after diagnoses to promote positive adjustment.

OBJECTIVE

The study protocol described here is part of the first phase of a larger program of research that aims to develop, adapt, and establish the feasibility of Resilient Together for Dementia (RT-ADRD), a novel dyadic skills-based intervention to be delivered over live video early after diagnosis, with the goal of preventing chronic emotional distress. This study will elicit and systematically summarize perspectives of ADRD medical stakeholders to inform the procedures (eg, recruitment and screening methods, eligibility, timing of intervention, and intervention delivery) of the first iteration of RT-ADRD prior to pilot-testing.

METHODS

We will recruit interdisciplinary medical stakeholders (eg, neurologists, social workers, neuropsychologists, care coordinators, and speech language pathologists) from academic medical center clinics in the departments treating persons living with dementia such as neurology, psychiatry, and geriatric medicine via flyers and word-of-mouth referrals from clinic directors and members of relevant organizations (eg, dementia care collaboratives and Alzheimer disease research centers). The participants will complete electronic screening and consent procedures. Consenting individuals will then participate in a 30- to 60-minute qualitative virtual focus group, held either via telephone or Zoom, using an interview guide designed to assess provider experiences with postdiagnosis clinical care and to gather feedback on the proposed RT-ADRD protocol. The participants will also have the opportunity to participate in an optional exit interview and web-based survey to gather additional feedback. Qualitative data will be analyzed using a hybrid inductive-deductive approach and the framework method for thematic synthesis. We will conduct approximately 6 focus groups with 4-6 individuals in each group (maximum N=30 individuals; until saturation is reached).

RESULTS

Data collection began in November 2022 and will continue through June 2023. We anticipate that the study will be completed by late 2023.

CONCLUSIONS

The results from this study will inform the procedures of the first live video RT-ADRD dyadic resiliency intervention focused on the prevention of chronic emotional and relational distress in couples shortly after ADRD diagnoses. Our study will allow us to gather comprehensive information from stakeholders on ways to best deliver our early prevention-focused intervention and gain detailed feedback on study procedures prior to further testing.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/45533.

Deployment of Real-time Natural Language Processing and Deep Learning Clinical Decision Support in the Electronic Health Record: Pipeline Implementation for an Opioid Misuse Screener in Hospitalized Adults

BACKGROUND

The clinical narrative in electronic health records (EHRs) carries valuable information for predictive analytics; however, its free-text form is difficult to mine and analyze for clinical decision support (CDS). Large-scale clinical natural language processing (NLP) pipelines have focused on data warehouse applications for retrospective research efforts. There remains a paucity of evidence for implementing NLP pipelines at the bedside for health care delivery.

OBJECTIVE

We aimed to detail a hospital-wide, operational pipeline to implement a real-time NLP-driven CDS tool and describe a protocol for an implementation framework with a user-centered design of the CDS tool.

METHODS

The pipeline integrated a previously trained open-source convolutional neural network model for screening opioid misuse that leveraged EHR notes mapped to standardized medical vocabularies in the Unified Medical Language System. A sample of 100 adult encounters were reviewed by a physician informaticist for silent testing of the deep learning algorithm before deployment. An end user interview survey was developed to examine the user acceptability of a best practice alert (BPA) to provide the screening results with recommendations. The planned implementation also included a human-centered design with user feedback on the BPA, an implementation framework with cost-effectiveness, and a noninferiority patient outcome analysis plan.

RESULTS

The pipeline was a reproducible workflow with a shared pseudocode for a cloud service to ingest, process, and store clinical notes as Health Level 7 messages from a major EHR vendor in an elastic cloud computing environment. Feature engineering of the notes used an open-source NLP engine, and the features were fed into the deep learning algorithm, with the results returned as a BPA in the EHR. On-site silent testing of the deep learning algorithm demonstrated a sensitivity of 93% (95% CI 66%-99%) and specificity of 92% (95% CI 84%-96%), similar to published validation studies. Before deployment, approvals were received across hospital committees for inpatient operations. Five interviews were conducted; they informed the development of an educational flyer and further modified the BPA to exclude certain patients and allow the refusal of recommendations. The longest delay in pipeline development was because of cybersecurity approvals, especially because of the exchange of protected health information between the Microsoft (Microsoft Corp) and Epic (Epic Systems Corp) cloud vendors. In silent testing, the resultant pipeline provided a BPA to the bedside within minutes of a provider entering a note in the EHR.

CONCLUSIONS

The components of the real-time NLP pipeline were detailed with open-source tools and pseudocode for other health systems to benchmark. The deployment of medical artificial intelligence systems in routine clinical care presents an important yet unfulfilled opportunity, and our protocol aimed to close the gap in the implementation of artificial intelligence-driven CDS.

TRIAL REGISTRATION

ClinicalTrials.gov NCT05745480; https://www.clinicaltrials.gov/ct2/show/NCT05745480.

The Feasibility of Providing Remote Functional Family Therapy with Adolescents During the COVID-19 Pandemic: A Mixed-Method Study

Background

Due to the recent COVID-19 pandemic, mental health care has largely transferred its services to online platforms, using videoconferencing (VC) or teletherapy. Within the field of family therapy, however, there is little evidence on the feasibility of using VC, especially when working with whole families at the edge of care.

Objective

This study investigated the feasibility of remote Functional Family Therapy (FFT), using a mixed-method approach.

Method

Study 1 consisted of semi-structured interviews with 23 FFT professionals (18 female) about their experience of providing remote FFT during the COVID-19 pandemic. Study 2 included monitoring data of 209 FFT clients (46% female, M _age  = 14.00) who participated in FFT during the pandemic. We compared families who received mainly in-person, mainly remote or a mix of remote and in-person on client-reported alliance, drop-out, therapist-rated outcomes, and treatment intensity using MANCOVA's and chi-square tests.

Results

In Study 1 two themes emerged around experienced challenges, namely 'Feeling in control' and 'Engagement and alliance'. Two other themes emerged around adaptations, namely 'Being more on top' and 'Connecting in different ways'. In Study 2, we found that the therapeutic alliance was not related to using VC. Also, families had less between-session contact during the Engagement and Motivation Phase when receiving mainly VC, but had more sessions and longer therapy when receiving a mix of in-person and remote therapy.

Conclusions

The current study suggests that providing systemic family teletherapy to families on the edge of care is feasible. Further development of systemic family teletherapy is warranted.

Supplementary Information

The online version contains supplementary material available at 10.1007/s10566-022-09692-y.

Dissemination and implementation science resources, training, and scientific activities provided through CTSA programs nationally: Opportunities to advance D&I research and training capacity

Introduction

Clinical and Translational Science Award (CTSA) Program hubs are well-positioned to advance dissemination and implementation (D&I) research and training capacity nationally, though little is known about what D&I research support and services CTSAs provide. To address this gap, the CTSA Dissemination, Implementation, and Knowledge Transfer Working Group conducted an environmental scan of CTSAs (2017-2018).

Methods

Of 67 CTSA institutions, we contacted 43 that previously reported delivering D&I research services. D&I experts from these institutions were emailed a survey assessing D&I resources, services, training, and scientific projects. Responses were categorized and double-coded by study authors using a content analysis approach.

Results

Thirty-five of the 43 D&I experts (81.4%) responded. Challenges to CTSAs in developing and supporting D&I science activities were related to inadequate D&I science workforce (45.7%) and lack of understanding of D&I science (25.7%). Services provided included consultation/mentoring programs (68%), pilot funding/grants (50%), and workshops/seminars/conferences (46%). Training and workforce development in D&I were frequently identified as future priorities. Recommendations included increase training to meet demand (68.6%), accessible D&I tools/resources (34.3%), greater visibility/awareness of D&I methods (34.3%), consultation services (22.9%), and expand D&I science workforce (22.9%).

Conclusions

CTSAs have tremendous potential to support the advancement and impact of D&I science across the translational continuum. Despite the growing presence of D&I science in CTSAs, continued commitment and prioritization are needed from CTSA and institutional leadership to raise awareness of D&I science and its value, meet training demands, and develop necessary infrastructure for conducting D&I science.

Applying the lessons of implementation science to maximize feasibility and usability in team science intervention development

The Science of Team Science (SciTS) has generated a substantial body of work detailing characteristics of effective teams. However, that knowledge has not been widely translated into accessible, active, actionable, evidence-based interventions to help translational teams enhance their team functioning and outcomes. Over the past decade, the field of Implementation Science has rapidly developed methods and approaches to increase the translation of biomedical research findings into clinical care, providing a roadmap for mitigating the challenges of developing interventions while maximizing feasibility and utility. Here, we propose an approach to intervention development using constructs from two Implementation Science frameworks, Consolidated Framework for Implementation Research, and Reach, Effectiveness, Adoption, Implementation, and Maintenance, to extend the Wisconsin Interventions for Team Science framework described in Rolland et al. 2021. These Implementation Science constructs can help SciTS researchers design, build, test, and disseminate interventions that meet the needs of both adopters, the institutional leadership that decides whether to adopt an intervention, and implementers, those actually using the intervention. Systematically considering the impact of design decisions on feasibility and usability may lead to the design of interventions that can quickly move from prototype to pilot test to pragmatic trials to assess their impact.

Enhancing translational team effectiveness: The Wisconsin Interventions in Team Science framework for translating empirically informed strategies into evidence-based interventions

Achieving the clinical, public health, economic, and policy benefits of translational science requires the integration and application of findings across biomedical, clinical, and behavioral science and health policy, and thus, collaboration across experts in these areas. To do so, translational teams need the skills, knowledge, and attitudes to mitigate challenges and build on strengths of cross-disciplinary collaboration. Though these competencies are not innate to teams, they can be built through the implementation of effective strategies and interventions. The Science of Team Science (SciTS) has contributed robust theories and evidence of empirically-informed strategies and best practices to enhance collaboration. Yet the field lacks methodological approaches to rigorously translate those strategies into evidence-based interventions to improve collaborative translational research. Here, we apply lessons from Implementation Science and Human-Centered Design & Engineering to describe the Wisconsin Interventions in Team Science (WITS) framework, a process for translating established team science strategies into evidence-based interventions to bolster translational team effectiveness. To illustrate our use of WITS, we describe how University of Wisconsin's Institute for Clinical and Translational Research translated the existing Collaboration Planning framework into a robust, scalable, replicable intervention. We conclude with recommendations for future SciTS research to refine and test the framework.