Combining Improvement and Implementation Sciences and Practices for the Post COVID-19 Era

Bridging clinical informatics and implementation science to improve cancer symptom management in ambulatory oncology practices: experiences from the IMPACT consortium

Objectives

To report lessons from integrating the methods and perspectives of clinical informatics (CI) and implementation science (IS) in the context of Improving the Management of symPtoms during and following Cancer Treatment (IMPACT) Consortium pragmatic trials.

Materials and Methods

IMPACT informaticists, trialists, and implementation scientists met to identify challenges and solutions by examining robust case examples from 3 Research Centers that are deploying systematic symptom assessment and management interventions via electronic health records (EHRs). Investigators discussed data collection and CI challenges, implementation strategies, and lessons learned.

Results

CI implementation strategies and EHRs systems were utilized to collect and act upon symptoms and impairments in functioning via electronic patient-reported outcomes (ePRO) captured in ambulatory oncology settings. Limited EHR functionality and data collection capabilities constrained the ability to address IS questions. Collecting ePRO data required significant planning and organizational champions adept at navigating ambiguity.

Discussion

Bringing together CI and IS perspectives offers critical opportunities for monitoring and managing cancer symptoms via ePROs. Discussions between CI and IS researchers identified and addressed gaps between applied informatics implementation and theory-based IS trial and evaluation methods. The use of common terminology may foster shared mental models between CI and IS communities to enhance EHR design to more effectively facilitate ePRO implementation and clinical responses.

Conclusion

Implementation of ePROs in ambulatory oncology clinics benefits from common understanding of the concepts, lexicon, and incentives between CI implementers and IS researchers to facilitate and measure the results of implementation efforts.

Comparing the CFIR-ERIC matching tool recommendations to real-world strategy effectiveness data: a mixed-methods study in the Veterans Health Administration

BACKGROUND

Practical and feasible methods for matching implementation strategies to diagnosed barriers of evidence-based interventions in real-world contexts are lacking. This evaluation compared actual implementation strategies applied with those recommended by an expert opinion-based tool to improve guideline-concordant cirrhosis care in a Veterans Health Administration national learning collaborative effort.

METHODS

This convergent parallel mixed-methods study aimed to (1) identify pre-implementation Consolidated Framework for Implementation Research (CFIR) barriers to cirrhosis care through focus groups with frontline providers, (2) generate 20 recommended strategies using focus group identified barriers entered into the CFIR-Expert Recommendations for Implementing Change (ERIC) Implementation Strategy Matching Tool, (3) survey providers over two consecutive years on the actual use of 73 ERIC strategies and determine strategy effectiveness, (4) compare actual versus recommended strategy use, and (5) compare actual versus expected barriers by reverse applying the CFIR-ERIC Matching Tool.

RESULTS

Eighteen semi-structured focus groups were conducted with 197 providers representing 95 VA sites to identify barriers to quality improvement, including cirrhosis care complexity, clarity of national goals, and local leadership support. The CFIR-ERIC Matching Tool recommended strategies such as assessing for readiness and needs, promoting adaptability, building local groups, preparing champions, and working with opinion leaders and early adopters. Subsequent strategy surveys found that sites used the top 20 "recommended" strategies no more frequently than other strategies. However, 14 (70%) of the top recommended strategies were significantly positively associated with cirrhosis care compared to 48% of actual strategies. Reverse CFIR-ERIC matching found that the strategies most used in the first year corresponded to the following barriers: opinion leaders, access to knowledge and information, and resources. The strategies most frequently employed in the second year addressed barriers such as champions, cosmopolitanism, readiness for implementation, relative priority, and patient needs and resources. Strategies used in both years were those that addressed adaptability, trialability, and compatibility.

CONCLUSIONS

This study is among the first to empirically evaluate the relationship between CFIR-ERIC Matching Tool recommended strategies and actual strategy selection and effectiveness in the real world. We found closer connections between recommended strategies and strategy effectiveness compared to strategy frequency, suggesting validity of barrier identification, and application of the expert-informed tool.

Learning from national implementation of the Veterans Affairs Clinical Resource Hub (CRH) program for improving access to care: protocol for a six year evaluation

BACKGROUND

The Veterans Affairs (VA) Clinical Resource Hub (CRH) program aims to improve patient access to care by implementing time-limited, regionally based primary or mental health staffing support to cover local staffing vacancies. VA's Office of Primary Care (OPC) designed CRH to support more than 1000 geographically disparate VA outpatient sites, many of which are in rural areas, by providing virtual contingency clinical staffing for sites experiencing primary care and mental health staffing deficits. The subsequently funded CRH evaluation, carried out by the VA Primary Care Analytics Team (PCAT), partnered with CRH program leaders and evaluation stakeholders to develop a protocol for a six-year CRH evaluation. The objectives for developing the CRH evaluation protocol were to prospectively: 1) identify the outcomes CRH aimed to achieve, and the key program elements designed to achieve them; 2) specify evaluation designs and data collection approaches for assessing CRH progress and success; and 3) guide the activities of five geographically dispersed evaluation teams.

METHODS

The protocol documents a multi-method CRH program evaluation design with qualitative and quantitative elements. The evaluation's overall goal is to assess CRH's return on investment to the VA and Veterans at six years through synthesis of findings on program effectiveness. The evaluation includes both observational and quasi-experimental elements reflecting impacts at the national, regional, outpatient site, and patient levels. The protocol is based on program evaluation theory, implementation science frameworks, literature on contingency staffing, and iterative review and revision by both research and clinical operations partners.

DISCUSSION

Health systems increasingly seek to use data to guide management and decision-making for newly implemented clinical programs and policies. Approaches for planning evaluations to accomplish this goal, however, are not well-established. By publishing the protocol, we aim to increase the validity and usefulness of subsequent evaluation findings. We also aim to provide an example of a program evaluation protocol developed within a learning health systems partnership.

Improve-mentation for Faster Testing and Spread of Health Service Delivery Innovations

Can we speed the testing, implementation and spread of management innovations in a systematic way to also contribute to scientific knowledge? Researchers and implementers have developed an approach to test and revise a local version of an innovation during its implementation. The chapter starts with a case example of an application of this combination of implementation and quality improvement sciences and practices (improve-mentation). It then summarizes four examples of this approach so as to help understand what improve-mentation is and how it is different from traditional quality improvement and traditional implementation of evidence-based practices. It considers gaps in knowledge that are hindering both more use of improve-mentation to generate scientific knowledge about spread and implementation, as well as more use of improve-mentation by health care service organizations and researchers. It closes by proposing fruitful research and development that can address these knowledge gaps to speed the implementation, sustainment and spread of care and management innovations.

Effect of COVID-19 on the implementation of a multifaceted intervention to improve teamwork and quality for hospitalized patients: a qualitative interview study

Background

Healthcare organizations made major adjustments to deliver care during the COVID pandemic, yet little is known about how these adjustments shaped ongoing quality and safety improvement efforts. We aimed to understand how COVID affected four U.S. hospitals’ prospective implementation efforts in an ongoing quality improvement initiative, the REdesigning SystEms to Improve Teamwork and Quality for Hospitalized Patients (RESET) project, which implemented complementary interventions to redesign systems of care for medical patients.

Methods

We conducted individual semi-structured interviews with 40 healthcare professionals to determine how COVID influenced RESET implementation. We used conventional qualitative content analysis to inductively code transcripts and identify themes in MAXQDA 2020.

Results

We identified three overarching themes and nine sub-themes. The three themes were (1) COVID exacerbated existing problems and created new ones. (2) RESET and other quality improvement efforts were not the priority during the pandemic. (3) Fidelity of RESET implementation regressed.

Conclusion

COVID had a profound impact on the implementation of a multifaceted intervention to improve quality and teamwork in four hospitals. Notably, COVID led to a diversion of attention and effort away from quality improvement efforts, like RESET, and sites varied in their ability to renew efforts over time. Our findings help explain how COVID adversely affected hospitals’ quality improvement efforts throughout the pandemic and support the need for research to identify elements important for fostering hospital resilience.

Interdisciplinary Approach and Patient/Family Partners to Improve Serious Illness Conversations in Outpatient Oncology

PURPOSE:

We aimed to increase Serious Illness Conversations (SIC) from a baseline of, at or near, zero to 25% of eligible patients by December 31, 2020.

METHODS:

We assembled an interdisciplinary team inclusive of a family partner and used the Model for Improvement as our quality improvement framework. The team developed a SMART Aim, key driver diagram, and SIC workflow. Standardized screening for SIC eligibility was implemented using the 2-year surprise question. Team members were trained in SIC communication skills by a trained facilitator and received ongoing coaching in quality improvement. We performed Plan-Do-Study-Act cycles and used audit-feedback data in weekly team meetings to inform iterative Plan-Do-Study-Act cycles. The primary outcome was the percent of eligible patients with documented SIC.

RESULTS:

Over 18 months, three clinics identified 63 eligible patients; of these, 32 (51%) were diagnosed with head and neck cancer and 31 (49%) with sarcoma. The SIC increased from a baseline near zero to 43 of 63 (70%) patients demonstrating three shifts in the median (95% CI). Conversations were interdisciplinary with 25 (57%) by oncology MD, six (14%) by advanced practice registered nurse, and 13 (30%) by specialty palliative care. We targeted four key drivers: (1) standardized work, (2) engaged interdisciplinary team, (3) engaged patients and families, and (4) system-level support.

CONCLUSION:

Our approach was successful in its documentation of end points and required resource investment (training and time) to embed into team workflows. Future work will evaluate scaling the approach across multiple clinics, the patient experience, and outcomes of care associated with oncology clinician–led SIC.